DOCUMENT RESUME ED 095 519 CS 001 329 SPONS AGENCY
Transcript of DOCUMENT RESUME ED 095 519 CS 001 329 SPONS AGENCY
DOCUMENT RESUME
ED 095 519 CS 001 329
AUTHOR Levie, W. Howard; Levie, Diane D.TITLE Is There a Separate Visual Iconic Memory System?
Final Report.INSTITUTION Indiana Univ., Bloomington. Audio-Visual Center.SPONS AGENCY National Inst. of Education (DHEW), Washington, D.C.
Office of Research Grants.BUREAU NO BR-3-0826-FRPUB DATE Aug 74GRANT NE-G-00-3-0107NOTE 54p.
EDRS PRICEDESCRIPTORS
MF-$0.75 HC-$3.15 PLUS POSTAGEAttention; Cognitive Processes; College Students;*Educational Research; Learning Processes; *Memory;Recall (Psychological); *Recognition; *Retention;*Visualization
ABSTRACTThe purpose of these studies was to provide evidence
to support either the dual-coding hypothesis or the single-systemhypothesis of human memory. In one experiment, college subjects wereshown a mixed series of words and pictures either whilesimultaneously engaged in shadowing (repeating aloud) a prose passagepresented via earphones or while free of such distractions. On amultiple-choice recognition test, it was found that the verbalslhadowing task interfered with word performance but failed to affectpictorial recognition memory, indicating that verbal information andpictorial information may be processed independently in parallelrecognition memory systems. In a similar experiment recall ratherthan recognition was the dependent measure. While the usual pictorialsuperiority was found, the verbal shadowing task interfered withpicture as well as word retention, suggesting that in memory taskswhere verbal reports are required, subjects characteristicallyattempt to translate easily labelable pictorial information intoverbal terms during the encoding phase. Thus there does appear to beseparate visual iconic memory system, but it normally operates in
total independence of the verbal system only when the anticipated useof information is nonverbal in character. (Author/WR)
el
N
U S DE' ARTMENT OF HEALTHEDUCATION &WELFARENATIONAL. INSTITUTE OF
EDO,:ATIONt 00( l %1 N RI 1 N st
1- x A( It t r,, Rt(1'- .E(11ROYI ra Pt 1410N OR of4,,,AN.e..s.ON 014.6,NA TiNt.. II POINTS OS ,t A OR OPINIONS',TAIL O DO NDT NU( t ',AWL Y RI 001SENT 01 I '('AL NAI ONA, IN.II TOTE 011 1 ) 1 1 ( A ) iON POS1 I ION C,Ft pot I Y
Final leport
Project No. 3-0826Grant No. NE-G-00-3-0107
Is There a SeparateVisual Iconic Memory System?
W. Howard Levieand
Diane D. Levie
Audio-Visual Center
Indiana UniversityBloomington, Indiana 47401
August, 19 74
The research reported herein was performed pursuant to a grant withtble National Institute of Education, U.S. Department of Health,Eeucation, and Welfare. Contractors undertaking such projects underGovernment sponsorship are encouraged to express freely their pro-fessional judgment in the conduct of the project. Points of viewor opinions stated do not, therefore, necessarily represent officialNational Institute of Education position or policy.
U.S. DEPARTMENT OFHEALTH, EDUCATION, AND WELFARENATIONAL INSTITUTE OP EDUCATION
OFFICE OP RESEARCH GRANTS
Abstract
Evidence was sought to support either the dual-coding hypothesisor single-system hypotheses of human memory. In one experiment, Sswere shown a mixed series of words and pictures either (1) whilesimultaneously engaged in shadowing (repeating aloud) a prose passagepresented via earphones or (2) free of such distraction. On a multiple-choice recognition test, it was found that the verbal shadowing taskinterfered with word performance but failed to affect pictorialrecognition memory, indicating that verbal information and pictorialinformation may be processed independently in parallel recognitionmemory systems. In another similar experiment, recall rather thanrecognition was the dependent measure. While the usual pictorialsuperiority was found, the verbal shadowing task interfered withpicture as well as word retention, suggesting that in memory taskswhere verbal reports are required, Ss characteristically attempt totranslate easily labelable pictorial information into verbal termsduring the encoding phase. Thus there does appear to be a separatevisual iconic memory system, but it normally operates in total in-dependence of the verbal system only when the anticipated use ofinformation is nonverbal in character.
Final Report
Project No. 3-0826Grant No. NE-G-00-3-0107
Is There a SeparateVisual Iconic Memory System?
W. Howard Levieand
Diane D. Levie
Audio-Visual CenterIndiana University
Bloomington, Indiana 47401
August, 1974
The research reported herein was performed pursuant to a grant withthe National Institute of Education, U.S. Department of Health,Education, and Welfare. Contractors undertaking such projects underGovernment sponsorship are encouraged to express freely their pro-fessional judgment in the conduct of the project. Points of viewor opinions stated do not, therefore, necessarily represent officialNational Institute of Education position or policy.
U.S. DEPARTMENT OFHEALTH, EDUCATION, AND WELFARENATIONAL. INSTITUTE OF EDUCATION
OFFICE OF RESEARCH GRANTS
Table of Contents
Contents
List of Tables
List of Figures
Introduction
"Single-System" Hypotheses 1
The Dual-Coding Hypothesis 2
Rationale and Outline of the Research 4
Experiment 1 6
Experiment 2 11
Experiment 3 16
Experiment 4 22
Summary and Conclusions 26
References 30
Appendix A: Word Pools for Experiment 1, 2, and 3 . 34
Appendix 8: Adjectives Used as Shadowing Material
in Experiment 1 35
Appendix C: Recognition Scores for Subjects in
Experiment 1 . . 36
Appendix D: Analysis of Variance Summary Table for
Experiment 1 37
Appendix E: Concepts Presented for "Animal, Vegetable,Mineral" Classification Task Used in Experiment 2 38
Appendix F: Recognition Scores for Subjects in
Experiment 2 39
Appendix G: Analysis of Variance Summary Table for
Experiment 2 40
Appendix H: Recognition Scores for Subjects in
Experiment 3 41
Content !NI
Appendix I: Analysis of Variance Summary Table forExperiment 3 42
Appendix J: Average Reported Confidence Levels inExperiment 3 43
Appendix K: Analysis of Variance Summary Table forAverage Reported Confidence Levels in Experiment 3 . 44
Appendix L: Concepts used in Experiment 4 45
Appendix M: Recall Scores for Subjects in Experiment 4 46
Appendix N: Analysis of Variance Summary Table forExperiment 4 47
List of Figures
Figure
1. Outcomes which would support eitherthe ducal- coding hypothesis or thesing' system hypothesis OOOOO
2. Percentage recognition accuracy for wor.and pictures under no shadowing andshadowing conditions
3. Average recall for words and picturesunder no shadowing and shadowingconditions
INTRODUCTION
When pictures are compared to words as stimuli in experimental
learning tasks, significant differences are usually found. Performance
on tasks involving pictures is almost always superior to that on tasks
involving words. Pictures result in better performance in recognitionmemory tasks (e.g., Shepard, 1967; Snodgrass, Volvovitz and Walfish,
1972; Standing, 1973)iin free recall tasks (e.g., Paivio, Rogers andSmythe, 1968; Kaplan. Kaplan and Sampson, 1968; Sampson, 1970), and asstimulus items in raired-associate learning tasks (e.g., Paivio and
Yarmey, 1966; Jenkins, 1968; Wicker, 1970). On the other hand, under
some conditions involving sequential memory, words result in better
performance (e.g., Paivio and Csapo, 1969, 1971; Del Castillo and
Gumentk, 1v72).
Researchers are discovering qualitative as well as quantitative
differences in the ways in which picture and words are processed by
the human information system. For example, while it is well known
that memory for words is highly dependent upon opportunities for
rehearsal, it appears that pictures are not ordinarily rehearsed butare recorded in memory more or less instantaneously (e.g., Shaffer
and Shiffrin, 1972; Hintzaan and Rogers, 1973; Cohen, 1973). Such
research has led to divergent hypotheses concerning the mechanisms by
which pictures and words are encoded, stored and retrieved in memory.
"Single-S7stem" Hypotheses
One tendency is to assort that the processing of pictorial stimuli
can be accounted for by the same memory mechanisms that are used to
describe the processing of verbal information.
The most straightforward of these explanations is the verbal
loop hypothesis. Glanzer and Clark (1963a, 1963b, 1964) propose thatpeople covertly translate pictorial information into a series of words,
store this verbalization, and retrieve from it when a response is
required. The type of evidence offered by these authors and others(e.g., Lantz and Steffre, 1964; Smith and Larson, 1970) in support of
the verbal loop hypothesis is that recognition accuracy of visual
displays such as color chips is related to subjects' verbal descriptions
of these displays and to the effectiveness of these descriptions in
communicating accurately.
As another example, picture-,word differences in discriminationlearning and recognition memory have been explained by a mechanism
that operates in the same manner for both words and pictures -- frequency
theory. In discrimination learning, previously seen "old" stimuli are
to be identified when presented with "now" stimuli added as distractors.
Frequency theory states that It is the unequal frequency of occurrenceof the new and old stimuli that is largely responsible for the subject's
ability to discriminate the old item. Ghatala, Levin, and Wilder (1973)
2
suggest that *hilt: the words used as stimuli in dit:rimination experimentshave been proviously encountered by the subject, the pictures usedare almost certainly unique in the subject's experience and henceconstitute more stable subjective frequency units. In an experimentin which the apparent frequencies of words and pictures were equated(Levin, Ghatala and Wilder, 1974), the usual superiority of pictureswas eliminated.
Other researchers have emphasized similarities between apparentunderlying processes used in memory for sentences and pictures. Benjafieldand Doan (1971) assert that comparative sentences (e.g., A is larger thanB) are encoded in the same way that corresponding pictures a(e 4.,O, a
drawing showing two pencils, one sharper than the other) are encoded.Similarly, Clark and Chase (1972) in a study of how people comparesentences against pictures (e.g., the sentence "Star isn't above plus"and the picture "tft) offer the proposition that a common "interpretive"system is used for both linguistic and perceptual stimuli of this kind.They argue that "...psychologists must come to regard certain thoughtprocesses not simply as 'verbal' or 'perceptual' in nature, but as'cognitive.'" (p. 515)
The most complex and well articulated approach of this kind is a
model developed by Anderson and Bower (1973) in which both linguisticand perceptual information are handled by a single conceptual-propositionalsystem. This system, called HAM for human associative memory, "...supposesthat knowledge -- even knowledge that is derived from pictures or thatis used in generating images -- is always represented in the form ofabstract propositions a1 ut properties of objects and relations betweenobjects." (p. 452) Interestingly, Bower has in the past been heavilyinvolved in research dealing with mental imagery and quite recently(Bower, 1972) subscribed to the notion of an imaginal as well as averbal memory system. This dramatic reversal in viewpoint by apsychologist of Bower's stature should give pause to those whounhesitatingly accept multiple system memory models.
The Dual-Coding Hypothesis
The dual-coding hypothesis (Paivio, 1971) proposes two separatememory systems, one for verbal symbolic processes and another fornonverbal imagery processes. The two systems may operate independently,but they are also richly interconnected and often operate in conjunctionwith each other. Thus, while linguistic information may be handledsolely by the verbal system, words may also evoke mental images, thusactivating the imaginal system. Similarly, pictures in addition toiconic encoding can be labeled or can otherwise arouse the verbal system.
While the dual-coding hypothesis has additional implications, theaspect of the model which is of concern here is the notion that morethan one system is involved. What follows is a brief review of theresearch that tends to support the plausibility of two memory systems --
3
each with the capability for independent operation.
One class of such evidence comes from neurophysiological research.
Considerable evidence has been produced to indicate that the left and
right cerebral hemispheres function differentially with respect to
verbal and imaginal processes (e.g., Sperry, 1961; Milner, 1970;
Geffen, Bradshaw and Wallace, 1971),. The left hemisphere dominates for
tasks which involve abstract thinking and linguistic processes while
the right hemisphere dominates for perceptual and nonverbal functions.
Researchers such as Seamon Gazzaniga (1973) argue that these brain
laterality effects suggest sepirate information systems. In related
research, Galin and Ornstein 11J72) found that sabjects who were engaged
in verbal cognitive operations uisplayed distinctly different EEG
patterns from subjects engage in spatial cognitive operations.
A variety of behavioral research appears to support the dual-system
hypothesis. Bahrick and Boucher (1968) found that verbal recall of the
word label for a simple drawing was unrelated to the probability of
correct visual recognition of the drawing. In a subsequent study,
Bahrick and Bahrick (1971) found no correlation between performance
on visual and verbal recognition tests of previously seen drawings.
Cermak (1971) provided evidence to support the existence of an
Independent visual ionic memory system by demonstrating that subjects
were able to recognize free-form nonsense figures under a condition
which would seen to preclude any verbal coding of the visual information.
Similarly, Paivio and Csapo (1969) demonstrated memory for pictures
which were presented at rates too rapid to allow for verbal coding.
Nelson and Brooks (1973) showed that while paired associate learning was
inhibited when word pairs were phonetically similar, there was no
difference in the learning of phonetically similar and phonetically
dissimilar picture pairs, implying that the pictorial code functioned
independently of the verbal label code. In a recognition test of
previously seen words and easily labeled pictures, Snodgrass, Wasser,
Finkelstein and Goldberg (1974) showed 0-4t subjects were later able to
identify whether a concept had been presented as a word or a picture,
syggesting that the two were encoded and stored differently.
Another group of behavioral researchers (den Heyer and Barrett,
1971; Meudell, 1972; Murray and Newnan, 1973) employed the following
basic scheme: (1) present a visual display containing both verbal
information and spatial information (in the case of den Heyer and
Barrett, letters placed in some of the boxes of a 4 x 6 matrix), (2)
during a retention interval require the subject to engage in either a
verbal activity or a visual activity (in the case of den Heyer and
Barrett,, an addition task or a visual discrimination task) and (3) test
for retention of the information in the display (the identity of the
letters and the location of the boxes in which they were placed). If
there are separate memory systems, an interaction between type of
information and type of distraction activity should result. The verbal
distraction task should interfere more with the rehearsal and retention
of the verbal informatiom,and the visual distraction should interfere
more with the rehearsal and retention of the nonverbal information.
4
Although there were minor variations in the three studies, the predicted
interaction was found in each case. In a study involving an imaginal
task or a linguistic task followed by visual or auditory interference,
similar results obtained (Atwood, 1971). Finally, Ternes and Yuille (1972)
found that a period of verbal interference (counting backwards by
three's) following the presentation of words and pictures depressed
recognition memory for words but not for pictures.
Rationale and Outline of the kesearch
The purpose of the research reported herein was to produce evidence
that would lend support to one or the other of the two rival hypotheses.
The basic scheme was to show subjects a series of words and pictures
intermixed. During the presentation, either (1) subjects were
simultaneously engaged in an auditory verbal task designed to "tie up"
the verbal processing system, or (2) they viewed the words and pictures
free of any distraction. If a test of retention were to show that the
verbal interference resulted in a decrwment in the learning of words while
not affecting the learning of pictures, this could be interpreted as
evidence of independent processing and support for the dual-system
hypothesis. On the other hand, if the verbal interference were to result
in similar effects on words and pictures, this could be interpreted as
evidence that the two are processed in the same manner as sugosted in
single-system hypotheses. (See Figure 1.)
Hi
An outcome which would support the An outcome which would support
dual-coding hypothesis
0
X
N
A
the single-system hypothesis
N
A 4 retention with no verbal interference
13 xetention during verbal interference
pictures
X - - X words
A
N.x
Figure 1: Outcomes which would support either the dual-coding
hypothesis or the single-system hypothesis.
5
It will be noted that this scheme bears a resemblance to theresearch reviewed in the preceding section dealing with the effectsof interference on the rehearsal and retention of verbal and nonverbal
information. An important difference in the present research is thatthe interference was introduced during the learning phase rather than
during the retention phase. This change is an improvement over pastresearch since, as was noted earlier, rehearsal is much more importantto the retention of verbal information than to the retention of
pictorial information. While it has been demonstrated that verbalinterference during rehearsal failed to hinder retention of pictorial
information (e.g., Ternes and Yuille, 1972), verbal interference duringthe learning phase might produce a deficit in performance.
The verbal interference used in the present research was the
technique known as "shadowing." In this technique, subjects are directed
to repeat aloud (shadow) spoken verbal material. Shadowing has been
shown to interfere with the learning of simultaneously presented verbalinformation (e.g., Norman, 1969). In the present research, the materials
to be learned were either pictures or nouns presented by projected slides
at two second intervals. Learning was tested by recognition in the first
three experiments and by free recall in the fourth experiment.
EXPERIMENT 1
Overview of Method
6
Experiment 1 was a recognition experiment. Each S was given two
trials, each of which was itself divided into two parts. During one
half of each trial, S was shadowing (listening to and repeating) a
list of recorded words; during the other half of each trial S was not
shadowing. The shadowing material used in one trial was composed of
a list of high-imagery adjectives and in the other trial of a list of
low-imagery adjectives. During each half of atrial, S was presented
visually with 40 slides, half of which were words and half of which
were pictures. At the conclusion of a trial, a recognition test was
given. S was presented pairs of slides side-by-side on the screen.
Each peUr consisted of either words or pictures. There were 40 pairs,
20 pairs of pictures and 20 pairs of words. In each pair, one slide had
appeared in the presentation series and one was a new slide (distractor
item). S was asked to identify the slide he had previously seen by
saying aloud "Left" or "Right." E recorded S's responses. S was
given as much time as necessary, Fut in no case was more than a few
seconds required.
Subjects
Sixteen students from graduate level courses in the Indiana University
School of Education volunteered to be subjects.
Materials
Stimulus materials consisted of 35mm slides presented by a slide
projector (or in the case of the test materials, two slide projectors)
and of audiotape recordings to be used in those experimental conditions
involving shadowing tasks.
Visually presented material consisted of word and picture slides
selected randomly from pools. The pools of word slides included
60 concrete words and 60 abstract words taken from a published list
(Paivio, Mille, and Madigan, 1968). Words in the concrete word pool
had values of I > 6.50 and C > 6.90; words in the abstract word pool
had values of I t 3.00 and C ' 2.50 (See Appendix A), The pools of
picture slides included 60 pictures designated as low similarity pictures
and 60 pictures designated as high similarity pictures. The low
similarity pool was assembled as a result of E's sorting through several
thousand slides on file in the Indiana University Audio-Visual Center.
These were photographed for a variety of purposes (e.g., instruction,
public relations, vacation) and by a variety of individuals. They included
pictures of real world objects and people, drawings and paintings, abstract
diagrams, and so forth, and they were in both color and black-and-white.
E made a highly subjective effort to select slides thought to be interesting
and/or memorable. The high similarity pool was assembled by delegating
one graduate student in photography to "Co out in the world and take some
pictures of things which attract you." These pictures, then, had the
7
commonality of having been taken by one person with one purpose andone camera; the.,v were of real world objects in out, community kt one
time and all in color. Pictures yore excluded from both low and highsimilarity pools if they included recognizable words (e.g., a readablestreet sign), if they subjectively seemed recognizable as a specificplace in this community (e.g., local ice cream store window), if theysubjectively appeared to be of the same labeiable content as thatdenoted by words in the concrete word pool, or if they seemed subjectivelysimilar to other pictures already in the pool (e.g., not more than onepicture of an automobile was included).
The visual stimulus presentation consisted of two trials, each oftwo parts. Each part of each trial had 40 slides, 10 of which wereconcrete words and 10 of which were abstract words. Both parts ofone trial utilized high similarity pictures, and both parts of theother trial utilized low similarity pictures. Given that these conditionswere fulfilled, random selection from the pools was made for all slides.
After the two parts of a trial had been presented (one with andone without shadowing), a recognition test was given. This consistedof the simultaneous presentation of two slides, one of which wasChosen from the preceding stimulus presentation and one of which wasa distractor item never seen before. Given that they equally representedthe two parts of the trial and proportionately represented the variousslide types existent in the parts of the trial, the slides chosen fromthe previous presentation were randomly chosen. (For example, therewere 10 abstract words in the first part of a trial and 10 abstractwords in the second part of a trim. Hence, abstract stimulus wordschosen for testing would include 5 chosen at random from the first10 and 5 chosen at random from the second 10.) Distractor or new
items used in the test were chosen at random from the general pools.Each pool had 60 items; the two trials used 40 items and the two testsused the remaining 20.
The slides were presented during the stimulus presentation by anautomatic timer at 2 second intervals. Pairs of slides were presentedduring the test presentation by B at the rate of S response.
Auditory material for shadowing consisted of adjectives recordedat 1 second intervals by a male voice on two tapes. One tape consistedof high-imagery adjectives (I -value > 4.25 and Thorndike-Lorge Frequency> 100) and the other of low-imagery adjectives (I -value < 3.00 andThorndike-Lorge Frequency > SO) taken from an unpublished list providedby Paivio. These lists appear in Appendix B.
Procedure
Experiment 1 was conducted in conjunction with and used the samesubjects as a pilot recall study not reported here. Ss underwent
this pilot study first and were given instructions ( which includedshadowing practice) quite similar to those noted in the proceduressection of Experiment 4. The following instructions, then, were
8
those given at the completion of experiment 4.
Instructions
Now we come to Part II of the experiment. Here
you'll see slides of many different types ofpictures and of words -- all mixed up. Once
again some of the time while you are looking atslides, you will elso be shadowing. As before,
your job is to Iemember the slides you see, but
this time, after two sizeable groups of slides,you will be shown pairs of slides. One of each
pair will be a slide yolove seen before and onea new slide. Your job is to pick out the
old one. It is also v important that you
shadow accurately.
Subsequent instructions depended on whether S was to shadow first or
second.
Por Ss who were to shadow first the instructions continued as
follows:
Again you'll wear the earphones throughout theentire experiment although you'll only be hearingwords and shadowing part of the time. When youput the earphones on and look at the screen, Iwill start tape recorder. After I have heard you
shadow a few words, I'll start the slides. Ready?
(After 2 words had been shadowed, E $gi thevisual display. The tape was continued andS shadowed for 8 additional words after thevisual display was concluded.)
Now I will show you another group of slides. You
will not be shadowing during this group. Justlook at the screen and try to remember the slides
you see.
(E presented the visual display.)
Now you will see pairs of slides. One will always
be a slide you've seen before and one will alwaysbe a new slide. Your job is to tell me which
slidejOn've seen before. Just say "Left" or
"Right" for the old slide. If you don't know,
make your best guess.
9
(E presented pairs of slides at S response rate
and recorded S's responses on a mimeographed
form.)
During this trial, S saw either all "high-similarity" or all "low-
similarity" pictures. Next, the procedure was repeated, and S saw
the type of picture not shown in the first trial. In order to control
for any possible order effects of (1) the shadowing condition, (2)
the high-low picture similarity condition, or (3) the high-low imagery
value of the shadowing material condition, all these conditions werebalanced and Ss randomly assigned to a given combination of conditions.
Results
The results of primary interest are shown in Table 1. The
shadowing activity had no effect upon the recognition of either
words or pictures. The recognition accuracy for the two types of
material is consistent with that found by other researchers, and
the main effect for w9rds versus pictures was significant. None
of the other variablekshowed significant main effects nor were
there any significant interactions. The raw data and the analysis
of variance summary table appear in the, Appendix C and Appendix D.
Tab le I
Recognition Scores and Percentage Accuracyfor Words and Pictures
under Adjective Shadowing and No Shadowing Conditions
ShadowingCondition
No ShadowingCondition
Words Pictures
246 302
(77%) (94%)
...-----
243 306
(76%) (95%)
Discussion
Two of the failures to produce differences are of interest. The
first of these is the lack of a difference between the recognition of
concrete words (78% accuracy) and abstract words (74% accuracy). An
analysis of variance for only the words in the study resulted in
a p value of about .25. This finding fails to support research by
Gorman (1961) and Atkinson and Juola (1973) who report better recognition
for concrete words than for abstract words.
10
The second item of interest is that the high-similarity and low-similarity pictures were recognized with equal accuracy -- in fact
with exactly equal accuracy, there being a total of 304 correctrecognitions in 320 opportunities for both sets of pictures. This
result was somewhat surprising to the authors since it had subjectively
appeared to them that the low-similarity set was on the whole a more
interesting group of pictures with less interstimulus similarity.Research by Standing (1973) shows that a set of pictures selected for
its vividness was more recognizable than a set which was not. Other
researchers have shown that recognition rates are depressed when there
is considerable interstimulus similarity in the stimulus population
(Goldstein and Chance, 1970). The present research indicates that
these pictorial stimulus variables will have effect on recognition memory
only when their differences are maximized.
The especially surprising (and especially disappointing) finding
was that the shadowing activity had no effect on either word or picture
recognition memory. This result was in contradiction of a short pilot
study conducted by the authors and of all previous research with
which the authors are familiar. The surprise, of course, is that the
verbal shadowing task failed to inhibit learning of words. Apparently
the rate of presentation of the shadowing material -- one word per second
-- was slow enough to allow for processing of the visual material, even
though Ss seemed to "have their mouth's full" and occasionally had
difficulty shadowing accurately. In any event, the authors' next task
was clear. In order to provide any test of the central hypothesis, it
was necessary to find a shadowing task which would have some effect on
recognition memory of visually presented material.
11
EXPERIMENT 2
Overview of Method
In Experiment 2, Ss underwent three trials, each of which involvedequivalent sets of visually presented word and picture slides, but eachof which differed significantly from the others in the nature of thesimultaneous activity engaged in. These activities were (1) a word
shadowing task, (2) a prose shadowing task, and (3) a classificationtask. After each trial, a recognition test involving sequentialpresentation of one old slide (from the stimulus presentation) and onenew slide (a distractor item) was given. S was asked to say "First"
or "Second" to indicate which slide he had seen before.
Subjects
The 12 Ss, graduate students in instructional systems technologyin the Education Department of Indiana University, volunteered to besubjects.
Materials
Stimulus materials consisted of 35mm slides visually presented by
a slide projector and of tape recordings auditorily presented by
earphones.
Three equivalent sets--each of 32 pictures and 32 words--wererandomly selected from two pools of slides. The picture pool includedthe 60 "high similarity" pictures and the 60 "low similarity" picturesof Experiment 1. In addition, 12 "high similarity" and 12 "lowsimilarity" pictures were added for a total picture pool of 144. The
high-low similarity sets had proved to be indistinguishable in theresults of Experiment 1; hence they were combined and treated as
equal members of one large set in Experimeet 2, i.e., simply ictures.
The word pool was composed of approximatbly half concrete and na tabstract words. These included the 60 concrete and 60 abstract wordsof Experiment 1 plus an additional 11 concrete and 13 abstract wordschosen in the same way as in the earlier experiment. Hence the total
word pool contained 144 words. Each of the three equivalent sets ofstimuli was composed of 32 randomly chosen pictures, 16 randomlyChosen abstract words, and 16 randomly chosen concrete words.
Audiotape recordings by the same male voice used in Experiment 1
were made to deliver the auditory tasks. The word shadowing taskconsisted of a combination of the high and low imagery adjectives usedin Experiment 1. The prose shadowing task consisted of a recordingof an excerpt from the Constitution of the United States (Article III,Sections 1 and 2). The classification task consisted of a list of
12
nouns recorded at two second intervals. Each noun was one readily
classifiable as "animal," "plant," or "mineral." (See Appendix E.)
For example, three of the words on the list are "goose," "daffodil,"
and "emerald." Upon hearing these words, S was to respond "animal,"after the first word, "plant" (or "vegetable") after the second werd,
and "mineral" after the third word.
Procedure
Experiment 2 involved 12 Ss, eachinitial instructions:
This is an AV experiment and30 minutes. I will show youthis
of whom received the following
it should take aboutslides of words like
(E presented a sample "word" slide.)
and slides of pictures like this...
(E presented a sample "picture" slide.)
After showing you a tray of slides -- some words andsome pictures -- I will then show you pairs of slides,one after the other, and ask you to teirgwhichslide you've seen before. One of the pair willalways be a slide you've seen before and one willalways be a new slide. For example, which of these
s dis have you seen before--the first or the second?Wait until you've soon them both before you answer.
(E presented two "picture" !lidos, the second ofwhich was the same as that already presented toS.
Right. Altogether, I will show you three trays of
slides. With each tray, I will ask you to do some-thing else simultaneously while you are watching
slides.
S was then given either the instructions for the classification task
or the instructions for the first shadowing task (regardless of
whether it was a word or a prose shadowing task). The instructions
for the classification task were as follows:
For this tray of slides I would like you to simultaneouslyplay a sort of animal-plant-mineral game. On the
earphones, you will hear the name of either an
animal, a plant, or a mineral. Your task is to say
immediately which it is aloud. For example, on the
short practice list I will give you, the first word is
13
"shale," and you should immediately respond bysaying "mineral." The second word is "pepper,"and you should say 'plant," and the third wordis "buffalo," and you should say "animal."Please put on the earphones and try it. If you
should miss one, that's okay. Just wait for thenext word and continue.
The instructions for the first shadowing task were as following:
During this set of slides, I would like you toshadow the words you hear on the earphones.Repeat aloud the words you hear immediately.
(If prose was first, E demonstrated how toshadao.)
Now, you try it.
(If S had trouble, as was often the case whenthe prose shadowing task was first, the practicetape was backed up. Otherwise, practicecontinued to the end of the practice tape.)
Okay, it is important that you shadow accurately.I will not test you on these words you shadow.I will test you only on the slides. However, doyour very best to shadow the words accurately. If
you miss something, just "throw it away" and divein again.
When S received his second shadowing task, the following instructionswere given:
And the instructions for the second shadowing task were as
follows:
For this tray, I would like you to again shadowthe words you hear. Try this...
(S received a practice tape. E demonstrated
tie prose shadowing if it was second.)
Again, it is important that you do your very best toshadow accurately.
Each S received three trials. A trial consisted of the visualpresentation of a 64 item set of word and picture slides accompaniedby a tape recording involving one of three experimental conditions(word shadowing, prose shadowing, ur classification task). Therewere six possible orders for the three experimental conditions and
also for the three equivalent slide sets. Given that the requirementof two Ss per order was met, a given S was randomly assigned to one ofthe six experimental condition orders and one of the six slide setorders.
After looking at the slides and simultaneously performing the taskindicated by the auditory material, S was given a recognition test.This test involved the sequential presentation of two slides, oneof which he had not seen before (distractor). The position of thesetwo (first or second) was randomly determined. S indicated the"old" item by saying "First" or "Second." E recorded the answers.
Results and Discussion
The primary results are shown in Table 2. The main effect for wordsversus pictures was the only significant difference in the analysis ofvariance (see Appendix G). However, a post hoc t-test for the differencebetween the retention of words under the adjective shadowing and proseshadowing conditions proved to be significant at the .05 level. The
recognition rate of 60% for words under the prose shadowing conditionapproaches the chance level of 50%. Also, it was very apparent duringthe administration that Ss were having the most difficulty in the prosecondition, and several a them commented to this effect.
Table 2
Recognition Scores and Percentage Accuracyfor Words and Pictures under Three Shadowing Conditions
AdjectiveCondition
ProseCondition
Anim-Veg-Min
Condition
Words Pictures
135
(70%)
178
(92%)
116 167
(60%) (87%)
128 172
(67%) (90%)
Parenthetically, a problem which arose during this study wasthe question of how to treat data obtained from Ss who did not shadow
errorlessly. It could be argued that an S who mime shadowing errors
WAS not attending to th' shadowing task, but was devoting his sole
attention at that moment to the visual material. Subjectively, this
did not appear to be the case during the administration. Ss who hdd
the most shadowing difficulty did appear to be trying and Were usually
the Ss who did poorest on the recognition test. In fact it could beargued that shadowing errors are preferred, since as Norman (1969) notes,errorless performance may mean that the S has spare capacity to process
the non-attended message.
16
EXPERIMENT 3
Although this experiment can be thought of as a replication ofExperiment 1, Experiment 3 differed from the first experiment in thefollowing important ways:
1. Ss shadowed prose material rather than the adjectives used inExperiment' 1. This change was introduced because of the results ofExperiment 2 showing prose to be more effective in depressing wordrecognition than the adjective shadowing task.
2. In Experiment 3, Ss were given more extensive training andpractice in shadowing prior to the experimental administration.
3. In the test phase, Ss were given four simultaneously presentedalternatives to choose from rather than two alternatives as inExperiment 1. This change would result in chance scores of 2S% ratherthan 50%, and offer the possibility of obtaining a wider spread inperformance scores. Kintsch (1968) showed that increasing the numberof alternatives in a multiple-choice recognition test decreasedperformance scores.
4. In the test phase, the type style of the words was different fromthe type style of the words in the presentation. The presentation wordswere a print style produced by a mechanical lettering device. Thetest words were an italic style produced by a typewriter. This changewas introduced to make recognition of the words depend upon verbal memory,not upon physical features of the verbal stimuli. An experiment had beenconducted earlier by the authors in which recognition of same-style wordswas compared with the recognition of different-style words. While theresults were in the predicted direction (same-style performance beingsuperior to different-style performance), the difference failed toreach significance. Other researchers, however, have obtained significantresults. For example, Kirsner (1973) used a continuous recognitionmemory test in which each word was presented twice, either in the sameprint or in different print an the two occasions. Results showedthat recognition performance was facilitated in the same print condition.Similarly, Kellicutt, Parks, Kroll, and Salzberg (1973) found that therecognition of letters was more rapid and accurate when test andpresentation letters were physically identical rather than the same inname only.
S. In Experiment 3, confidence ratings were also obtained. Aftereach response in the test phase, S indicated the degree of confidence hehad in the accuracy of his response by stating that he was "Very sure,""Moderately sure," "Unsure," or that his response was "Just a guess."
17
Overview of Method
Each of 16 Ss had two trials. In each trial, S saw a presentationof 60 slides, 30 of which were pictures and 30 of which were words.The two trials differed in that one involved no shadowing or auditorytask while the other trial required S to shadow prose material heardthrough earphones at the same time he was presented with slides tobe remembered for later recognition. Half of the Ss did theshadowing trial first; half did the nonshadowing trial first.
The words used in both the presentation phases and the test phaseswere half abstract and half concrete. The type style of the presentationwords was print style while the type style of the test words was italicstyle.
Immediately after each of the two presentations, S was given arecognition test on 20 items randomly selected from the prior presentationitems. (The selection was random given that the balance of picture,abstract word, and concrete word was the same as that in the presentation.)Each of the 20 test items involved 4 simultaneously presented slides-- -all pictures or all words---positioned in what might be described as a2 x 2 matrix. One of the four slides was always an old item, i.e., anitem previously seen during the presentation phase. Three of the
slides were always new or distractor items. Old or "correct" itemswere randomly located among the 4 possible positions. S was asked toindicate the old item by saying "Top left," "Top right,ff "Bottom left,""Bottom right." Those responses were recorded by the experimenter on amimeographed form. Also recorded by the experimenter were S confidenceratings on each item.
Sub ects
The 16 Ss were Indiana University students enrolled in graduatelevel courses in the School of Education. All volunterred to be Ss.
Half were female and half male.
Materials
The words and pictures used as presentation and test material camefrom the same pools as used in Experiment 2, with the previously notedexception of the test words. The auditory shadowing material was theaudiotape of a section of the U.S. Constitution used in Experiment 2.Additional readings from the Constitution were recorded by the "Jamevoice for use in the initial, extended practice phase.
18
Procedure
S was first shown samples of the presentation and test material,and instructed in how to report his responses. He was then told now to
report confidence ratings as follows:
After telling me which slide you think you saw before,I would like you to tell me how certain you are thatyou made the correct choice. Tell me by selectingone response from the card on the table beside you.
(The card listed the following responses: (1) "very
sure," (2) "mode-ately sure," (3) "unsure," (4) "just a
guess.") If you are very sure that you were able to pickout the slide you saw before, say "Very sure," and so forth.
If you really just can't tell which of the four slidesit was, make a guess anyway and tell me "Just a guess."
Ss were then instructed in the shadowing technique, after which they were
given two practice periods. The first period consisted of a one minute
section from the Constitution. The second practice period consisted of
a 45 second section. During this period a set of stimulus slidessimilar to those which would be used in the experiment were shown along
with the shadowing material. Ss consequently knew exactly what to
expect during the experiment. -.Following the practice, Ss were instructed,
(For S3 who shadowed first.)Now we will do it for real. Repeat the words asaccurately as you can and watch the slides carefully.Watch the screen and begin repeating as soon as you
hear the words begin.
About two seconds after S began to shadow, the visual stimuli began to
appear at two second intervals. After the last slide, the shadowing
material continued for about eight seconds. Prior to the testing, there
Was a pause of :bout 20 seconds while E readied the apparatus for
displaying the test slides. Following the test for Trial 1, S was
administered Trial 2, either not shadowing or shadowing, whichever
condition he did not have in the first trial.
Results
In Experiment 3, the main effects for shadowing condition and for
pictures versus words were both significant at the .01 level (see
Appendix I). More importantly, the interaction was also significant
at the .01 level. The shadowing condition depressed word performance
but ha( negligible effect upon pictorial recognition memory. (See Table 3.)
Table 3
Recognition Scores and Percentage Accuracyfor Words and Pictures
under Prose Shadowing and No Shadowing Conditions
ShadowingCondition
No ShadowingCondition
Words Pictures
70(44%)
---........._
149
(93%)
.
110 154
(69%) (96%)
19
The same significant differences that were produced for the
recognition data were found for the confidence ratings. Confidenceratings were higher for pictures than for words, higher for theno shadowing condition than for the shadowing condition, and the
interaction was significant. The shadowing task resulted in agreater drop in confidence for the words than for the pictures. The
confidence rating of "Very sure" was assigned a score of 3, "Moderatelysure" a score of 2, "Unsure" a score of 1, and "Just a guess" a scoreof 0, so that in Table 4, the average rating or words in the noshadowing condition was "Moderately sure" while the average rating forpictures under the same condition was midway between "Moderately sure"
and "Very sure." The shadowing task resulted in a drop of 0.3 points
for pictures and 0.7 points for words. While the reader is cautionedthat this rating scale should not be considered to be a strict intervalscale, the results could nevertheless be interpreted as indicating thatthe verbal interference produced a greater drop in confidence to retainand recognize verbal material than pictorial material.
Table 4
Average Confidence Ratingsfor Words and Pictures
under Prose Shadowing and No Shadowing Conditions
ShadowingCondition
No ShadowingCondition
Discussion
Words Pictures
1.3 2.2
2.0 2.5
20
The interaction between the shadowing conditions and the stimulus
types was earlier identified as the effect which would tend to support
the dual-encoding hypothesis. This interaction was obtained; verbal
interference (shadowing) produced a large decrement in word recognition
while having essentially no effect upon picture recognition.
It is interesting to compare the percentage accuracies in this
experiment with those in Experiment 1. Remember that in Experiment 3
not only was the shadowing task more difficult, but the test was also
more difficult, involving a choice from among four alternatives rather
than from two alternatives. Nevertheless, pictorial recognition accuracy
remained at a constant level. Word recognition accuracy suffered only
slightly in the no shadowing condition (69% with four alternatives and
76% with two alternatives in Experiment 1).
While the authors were conducting the present set of studies, a
similar experiment was reported in the literature. Rollins and
Thibadeau (1973) cosparad recognition of items in shadowing and no
shadowing conditions. They presented a prose passage via earphones
to one ear for the shadowing task. Ss were to learn either (1) words
presented auditorially via earphonesto the second ear, (2) words
presented visually, (3) pictures of common objects, or (4) pictures of
fictitious animal-like characters which were designed to be hard to
attach verbal labels to. Shadowing produced a significant decrement in
the recognition of the first three types of items, but not for the
fictitious characters. The authors state that "...attending to an
auditory message interferes with the processing and storage of any
information whether visually or auditorily presented when that information
can be verbally labeled," (p. 166)
21
The two conditions from the Rollins and Thibadeau study whichmost nearly match the present study are those involving visuallypresented words and pictures of common objects. The comparison of
greatest interst here is that involving pictures. Rollins and Thibadeau
obtained a rate of 94% in the no shadowing condition and 78% in the
shadowing condition. Numerous researchers have found that picturespresented sequentially at two second intervals with, no or negligableintervening "blank" time consistently result in recognition rates of
around 95%. This rate is comparable to the rate found in Experiment3 with or without shadowing, which has been interpreted herein tosuggest that picture memory is not affected by verbal shadowing tasks.
This rate is also comparable to that obtained by Rollins and Thibadeau
in their no shadowing condition (94%). However, Rollins and Thibadeau
used a one second presentation period, which in other studies (Potter
and Levy, 1969) resulted in performance rates of about 80%. The
explanation may lie in the fact that while Rollins and Thibadeau used aone second presentation, this period was followed by a 1.6 second
blank period during which time rehearsal could occur. Since other
researchers have indicated that picture memory does not benefit fromrehearsal, it might be assumed that the rehearsal advantage occurringhere, if indeed it is occurring, must be in a verbal form. That is to
say, in the Rollins and Thibadeau study, under the picture-no shadowingcondition, Ss used the blank period to recede the picture material into
a verbal form which they then rehearsed. This sane explanation could
also account for the reduced performance when picturee are accompaniedby shadowing, since under these circumstances verbal rehearsal is greatlyreduced by the shadowing task.
If the preceding explanation is viable, a modification of the
conclusion offered by Rollins and Thibadeau would be that attending to
an auditory message interferes with the processing and storage of
visually presented information if that information is verbally labeled.
22
EXPERIMENT 4
Would the results obtained in Experiment 3 be produced if the measureof retention were recall rather than recognition? Frost (1972) showedthat pictures are encoded differently depending upon task expectation.Tversky (1973) found that both words and pictures are encoded differentlydepending upon the expected use of the information. Performance wasbetter when Ss were tested in the way they were anticipating (eitherrecognition or recall) than when they were tested in the unanticipatedmode, suggesting the use of different encoding strategies during thelearning phase. Hence an experiment was conducted to see if shadowingwould have effects upon recall performance (and the strategies involved)that were similar to the effects upon recognition.
Overview of Method
Each S was given four trials. Each trial consisted of the presentation
of 12 slidii. In two trials the material-to-be learned was concrete
nouns and in two trials the material-to-be learned was line drawings.
During one of the word trials and one of the picture trials, S shadoweda section of the U.S. Constitution. After each trial, S wrote down all
of the items he could remember.
Subjects
The 16 Ss were graduate students in Indiana University School ofEducation. All volunteered to be Ss.
Materials
Stimulus materials consisted of 35mm slides presented by a slideprojector and an interval timer at a 2 second presentation rate andportions of the tape recordings used in Experiment 3.
Using as a general source a commercially prepared set of elementarylevel flashcards of concrete objects, a pool of 48 simple concepts was
developed (see Appendix L). A slide was prepared showing each conceptas a word and another slide was prepared showing the concept as a simple
line drawing. For each S a different random order of these 48 concepts
was devised. That is to say, whether a concept was presented as a wordor as a picture, and the order in which each concept was presented, wasdetermined randomly for each S. The order II which each S received
each treatment (whether word or picture, shadowing or not shadowing)was also randomly determined.
23
Procedure
Ss received the following instructions:
We are studying how peopleunder various conditions.will be showing you slides
remember visual displaysIn this experiment Ilike this.
(E presented sample word and picture slides.)
Some of the time you will be looking at pictures and
some of the time you will be looking at words. Also,
some of the time while you are looking at slides you
will also be shadowing, that is, listening to and
repeating words, like this.
gave a shot)* demonstration and then hadS put on earphones and try a short section.)
As you have probably guessed, these are some sections
of the U.S. Constitution. Do not worry about remembering
what you hear. I will not in any way test you aboutwhat you hear and repeat, but it is important that you
do your best to shadow accurately. This is not aparticularly easy thing to do at first, so I'd like
to give you a little practice. Here is an about one
minute section. Pace the screen, and do your best
to shadow accurately.
Following this practice session, Ss were shown how to provide
recall responses on a provided form, and were instructed to wait
until E instructed them to begin before writing down the items they
could recall. They were then given a practice trial with four slides,
two words and two pictures. After this practice trial, Ss were
instructed:
Right. If it's a picture, give the single word
that labels the picture. If it's a word, justgive the word, but remember to wait until I
say "begin." Altogether you will see four different
groups of slides. There will be 12 slides in
each group. Here is the form for the first
group. This group will be all (words, pictures),and you (will, will not) be shadowing. ON
SHADOWING TRIALS THE INSTRUCTIONS CONTINUED:Now look at the screen and when you hear the words on the
earphones, begin shadowing and continue shadowing
until the words end. Then when I say "begin" writedown as many slides as you can, and do your best to
shadow accurately. ON TRIALS WHEN S WAS NOT TO
SHADOW: Now look at the screen. 'Men afterwards when
I say "Begin" write down as many slides as you can.
24
During the shadowing trials the shadowing material began about
two seconds prior to the presentation of the first stimulus and continued
on for about eight seconds after the last slide, after which Ss were
immediately instructed to begin writing. In the non - shadowing trials,
Ss were instructed to begin writing after an eight second delay following
the last slide. Ss were given as long as they wished to recall, but in
no case did an S take more than three minutes. Typically Ss took about
one and a half minutes to respond. E removed the response form used
after each trial and gave S a new form.
Results and Discussion
In Experiment 4, the main effect for shadowing was significant(p < .01) as was the main effect for words versus pictures ( p < .05).
The interaction, however, failed to achieve significance (see Appendix
N). Table 5 shows the average number of concepts recalled out of apossible 12 presented in each of the four conditions. Shadowing
reduced recall of words by 56% and recall of pictures by 36%.
Table S
Average Number of Words and Pictures Recalled
under Prose Shadowing and No Shadowing Conditions
ShadowingCondition
No ShadowingCondition
Words Pictures
3.2
7.2
4.9
7.6
The failure to produce a significant interaction between stimulus
type and shadowing condition is not consistent with the results reported
by Hall and Swane (1973) and by Hall, Swan* and Jenkins (1973). In the
former study, color patches or color names were presented to Ss
shadowing words presented at the rate of two worde per second. A
test for recall in serial position showed better performance for color
patches than for color names. Although the design of the experiment
did not allow for the determination of a stimulus type by shadowing
condition interaction, a group of control Ss did equally well with
patches u with names. In the Hall, Swansend Jenkins (1973) study,similar results were obtained for "physical irputs" consisting of
rniletric figures varying in form, number of figures and color versusthe "semantic input" of the name of the figures (e.g., three black
circles).
25
Why was the interaction produced in Experiment 3 not found in
Experiment 4? It has been noted (Tversky, 1973) that recognition tests
may eliminate the retrieval stage of memory necessary in recall tests.
In pictorial recognition, it is a logical possibility that no verbal
transactions need be involved. The results of Experiment 3 are com-
patible with this point of view. In pictorial recall, however, a
verbal response is necessary; hence, some kind of verbal transaction
must be involved. Information processing strategies for pictorial
recall might consist of coring some sort of untranslated iconic
representation in memory, .And, when recall is demanded, searching this
storage, retrieving from It, and only then translating the iconic
storage into verbal form. However, such a description is not consistent
with the results of Experiment 4; verbal interference was present
during presentation, not during recall, and given the processing
strategies outlined above, should not then depress performance on
pictures. But, what if the nature of the experimental task (i.e.,
Ss'awareness that recall, not recognition, was the performance task
required) led Ss to translate the pictorial stimuli into words during
the presentation stage? This explanation would account for the depressed
performance on pictures presented under the shadowing condition and for
the absence of a significant interaction in Experiment 4.
SUMMARY AND CONCLUSIONS
ta_r_imaz
26
The purpose of this research was to provide evidence to support
either the dual-encoding hypothesis or single-system hypotheses of
human memory.
In single-system hypotheses the same mechanisms are used to
account for the processing of both verbal and pictorial stimuli.
Such explanations include the verbal-loop hypothesis (Glanzer and
Clark, 1964), the frequency theory account of picture-word differences
in discrimination learning (Levin, Ghatala, and Wilder, 1974), and
Anderson and Bower's (1973) single conceptual system in which knowledge
from both pictures and words is represented in the form of abstract
propositions. The dual- coding hypothesis (Paivio, 1971) proposes two
separate memory systems, one for verbal symbolic processes and another
for nonverbal imagery. While the two systems are thought to be
richly interconnected, it is supposed that they also may function
independently.
Experiments were conducted to test the independence of memory for
words and pictures. The basic parading was to compare the retention
of visually presented words and pictures under two conditions. In the
first condition, Ss were shown the vislal material while they were
simultaneously engaged in an auditory verbal task designed to "tie up"
the verbal processing system. In the second condition, Ss viewed the
words and pictures free of any distraction. If, under tie "verbal
interference" condition, performance were decreased for words but
not for pictures, this could be interpreted as an indication that
verbal and pictorial stimuli may be processed by separate memory
systems as proposed in the dual-coding hypothesis. If, on the other
hand, the "verbal interference" were to result in similar decrements
in picture performance and word performance, this could be interpreted
as lending support to single-system hypotheses of human memory.
Word and picture retention was assessed by recognition and by
recall in separate experiments. The first three experiments dealt with
recognition. Experiment 3 embodies the resolution of certain methodo-
logical problems existent in Experiments 1 and k; hence only Experiment
3 is reviewed below.
In Experiment 3, each of the 16 Ss was given two trials. Each
trial consisted of a mixture of 30 words (concrete and abstract nouns
on 35mm slides) and 30 pictures (vacation slides, instructional
illustrations, etc.) presented at two second intervals. During one
of the trials Ss also shadowed (repeated aloud) a prose passage presented
via earphones. Immediately after each of the two presentations, Ss
were given a recognition test on 10 words and 10 pictures from thi
prior presentation. Each "old" item was accompanied by three "new"
27
or distractor items. The type-style of the words shown during thelearning phase was different from the type-style of the words used inthe recognition test. This was done so that word recognition would bedependent upon verbal memory and not upon physical features of theverbal stimuli.
The main effects for words versus pictures and for shadowingconditions as well as the interaction were all significant (p < .01).As shown in Figure 2, the shadowing activity depressed word performancebut had negligible effect upon pictorial recognition memory.
pictures(93%)(96%) o o
(69%)
PercentageRecognitionAccuracy
words
(44%)
No WithShadowing Shadowing
Figure 2: Percentage recognition accuracy for words and picturesunder no shadowing and shadowing conditions.
This evidence corrobrates earlier research showing that verbalactivity such as counting backwards by three's during a retentionperiod produced more interference with retention of verbal materialthan of nonverbal material. However, since it has also been shownthat the retention of pictorial information is much less dependent uponrehearsal than the retention of verbal information, the present designwhich introduced the verbal interference during the learning phaserather than during the retention and rehearsal phase is thought to bean improvement over previous research of this kind.
28
Experiment 4 was conducted to see if the shadowing activity wouldproduce similar effects upon free recall. Each of the 16 Ss were given
four trials. Each trial consisted of 12 slides presented it two secondintervals. Two trials showed concrete nouns and two showed simple line
drawings. (The 48 concepts used were prepared in both word and pictureform.) During one of the word trials and one of the picture trials Ssshadowed prose material of the type used in Experiment 3. After eadE.
zrial Ss wrote down as many of the stimuli as they could recall.
AverageNumber ofStimuliRecalled
(7.6)
(7.3)
(4.9)
(3,.2)
Shadowing
n
Shadowing
Figure 3: Average recall for words and pictures under noshadowing and shadowing conditions
The results showed a main effect for words versus pictures (p < .05)
and for shadowing conditions (p < .01). The interaction between stimulus
type and shadowing condition was not significant. As shown in Figure 3,the shadowing activity disrupted recall of both picture and word stimuli.
The authors speculate that the failure to obtain an interaction similarto that found with the recognition data was due to the likelihood that
Ss attempted to translate the easily labelable pictures into wordsduring the presentation stage and that this process was disrupted by
the shadowing task.
Conclusions
Is there a separate visual iconic memory system? Yes. The results
of Experiment 3 provide strong evidence that verbal and pictorialinformation may be processed independently in parallel recognition
memory systems. The finding that the verbal shadowing task produced alarge decrement in word recognition while resulting in essentially nodecrement in pictorial recognition memory is very striking.
29
The results of Experiment 4 serve to emphasize that the independence
of the verbal and nonverbal systems may be preserved only when no
verbal transactions are necessary. Even though material to be learned
is presented in pictorial form, it cannot be guaranteed that the
information will be processed primarily or even partially by the
visual iconic component of memory. Under some circumstances, Ss may
"instruct themselves" to reply primarily upon verbal translations of
pictorial information in memory tasks.
While there is always danger in extrapolating results from
laboratory experiments to prescriptions for instructional practice,
some implications may be offered with an unusually high degree of
confidence in this case. The experimental conditions in which
meaningful pictures and words were presented in conjunction with
non-redundent meaningful prose is analogous to many types of
audiovisual presentations used in education. Producers and users
of audiovisual materials such as sound motion pictures may be
advised that, under circumstances when the receiver (student) is
attending to verbal information in the sound track, (1) considerable
loss of information from non-redundant visually presented verbal
material may be expected, (2) considerable retention of pictorial
information may be expected if retention is measured by recognition,
and (3) it is probable that considerable loss of pictorial information
will occur if retention is tested by verbal report.
30
REFERENCES
Anderson, J. R., & Bower, G. J. Human associative memory. Washington,
D. C.: V. H. Winston . Sons, 1073.
Atkinson, R. C., & Juola, J. F. Factors influencing speed and accuracy
of word recognition. In S. Kornblum (Ed.), Attention and Performance
I, New York: Academic Press, 1973, pp. 583-612.
Atwood, G. An experimental study of visual imagination and memory.
Cognitive Psychology, 1971, 2, 290-299.
Bahrick, H. P., & Bahrick, P. Independence of verbal and visual codes of
the same stimuli. Journal of Experimental Psychology, 1971, 91,
344-346.
Bahrick, H. P., & Boucher, B. Retention of visual and verbal codes of
the same stimuli. Journal of Experimental Psychology, 1968, 78,
417-442.
Benjafield, J., & Dean, B. Similarities between memory for visually
perceived relations and comparative sentences. Psychomomic Science,
1971, 24, 255-256.
Bower, G. J. Mental imagery and associative learning. In L. Gregg (Ed.),
Cognition in learning and memory. New York: Wiley, 1972.
Cermak, G. W. Short-term recognition memory for complex free-form figures.
Psychonomic Science, 1971, 25, 209-211.
Clark, H. H., & Chase, W. G. On the process of comparing sentences against
pictures. Cognitive Psychology, 1972, 3, 472-517,
Cohen, G. How are pictures registered in rhemory? Quarterly Journal of
Experimental Psychology., 1973, 25, 557-564.
Del Castillo, D. M., & Gumenik, W. E. Sequential memory for familiar and
unfamiliar forms. 22kiyJournalofExeznentalPsycholo, 1972, 95,
90-96.
den Meyer, K., & Barrett, B. Selective loss of visual and verbal
information in STM by means of visual and verbal interpolated tasks.
Esychonomic Science, 1971, 25, 100-102.
vrost, N. Encoding and retrieval in visual memory tasks. Journal of
Everimontal Psychology, 1972, 95, 317-326,
Galin, D., 6 Ornstein, R. Lateral specialization of cognitive mode: An
study, Psychophysiology, 1972, 9, 412-418.
31
Geffen, G., Bradshaw, J. L., & Wallace, G. Interhemispheric effects
on reaction time to verbal and nonverbal stimuli. Journal of
Experimental Psychology!, 1971, 87, 415-422.
Ghatala, E. S., Levin, J. R., Wilder, L. Apparent frequency of
words and pictures as a function of pronunciation and imagery.Journal of Verbal Learnin: and Verbal Behavior, 1973, 12, 85-90.
Glanzer, J., & Clark, W. H. Accuracy of perceptual recall: An analysis
of organization. Journal of Verbal Learning and Verbal Behavior,1963, 1, 289-299. 010
Glanzer, M., & Clark, W. H. The verbal loop hypothesis: Binary numbers.
Journal of Verbal Learning and Verbal Behavior, 1963, 2, 301-309.lb)
Glanzer, M., & Clark, W. H. The verbal-loop hypothesis: Conventional
figures. American Journal of Psychologr, 1964, 77, 621-626.
Goldstein, A. G., & Chance, J. E. Visual recognition memory for complex
configurations. LLcIPsiaayPercetionwhhsics, 1970, 9, 237-241.
Gorman, A. M. Recognition memory for nouns as a function of abstractness
and frequency. Journal of Experimental Psychology, 1961, 61,
23-29.
Hall, R., & Swane, D. Memory in a bisensory selective attention task.Quarterly Journal of Experimental Psychology, 1973, 25, 309-315.
Hall, R., Swane, D., & Jenkins, R. A. Memory for physical and semantic
features of visual material in a shadowing task. Journal ofExperimental Psychology, 1973, 100, 426-428.
Hintzman, D. L., & Rogers, M. K. Spacing effects in picture memory.
Memory & Cognition, 1973, 1, 430-434.
Jenkins, J. R. Effects of incidental cues and encoding strategies onpaired-associate learning. Journal of Educational Psychology,
1968, 59, 410-413.
Kaplan, S., Kaplan, R., & Sampson, J. R. Encoding and arousal fectors
in free recall of verbal and visual material. Psychonomic Science,
1968, 12, 73-74.
Kellicutt, M. H., Parks, T. B., Kroll, N. E. A., h Salzberg, P. M. Visual
memory as indicated by latency of recognition for normal and reversed
letters. JotsyEnalofEnnentalPscholo, 1973, 97, 387-390.
Kintsch, W. An experimental analysis of single stimulus tests and
multiple-choice tests of recognition memory. Journal of Experimental
Psychology, 1968, 76, 1-6.
32
Kirsner, K. An analysis of the visual component in recognition memoryfor verbal stimuli. Memory & Cognition, 1973, 1, 449-453.
Lantz, D., & Steffire, V. Lanugage and cognition revisited. Journalof Abnormal and Social Psychology, 1964, 69, 472-481.
Levin, J. R., Ghatala, E. S., & Wilder, L. Picture-word differences indiscrimination learning: I. Apparent frequency manipulations.Journal of Experimental Psychology, 1974, 102, 691-695.
Meudell, P. R. Short-term visual memory: Comparative effects of twotypes of distraction on the recall of visually presented verbaland nonverbal material. Journal of Experimental Psychology, 1972,94, 244-247.
Milner, B. Memory and the medial temporal regions of the brain. In
K. H. Pribram and D. E. Broadbent (Eds.), Biology of memory.New York: Academic Press, 1970.
Murray, D. J., & Newman, F. M. Visual and verbal coding in short-termmemory. Journal of Experimental Psychology, 1973, 100, 58-62.
Nelson, D.their1973,
L., Brooks, D. H. Functional independence of pictures andverbal memory codes. Jour)alofEezg12LaItalPscholo,98, 44-48.
Norman, D. A. Memory while shadowing. Quarterly Journal of Psychology,1969, 21, 85-93.
Paivio, A. Imagery_ and verbal processes. New Yoik: Holt, Rinehartand Winston, 1J71.
Paivio, A., & Csapo, K. Concrete-image and verbal memory codes. JournalaltaIxofEerimeizholo, 1969, 80, 279-285.
Paivio, A.,words.
Paivio, A.,recall
Csapo, K. Short-term sequential memory for pictures andPsychonomic Science, 1971, 24, S0 -51.
Rogers, T. B., $ Smythe, P. C. Why are pictures easier tothan words? Psychonomic lcienti. 1968, 11, 137-138.
Paivio, A., & Yarmey, A. D. Pictures versus words as stimuli and responsesin paired-associate learning. plychonomic Science, 1966, 5, 235-236.
Paivio, A., Yuille, J. C., 8 Madigan, S. Concreteness, imagery, andmeaningfulness values for 925 nouns. Journal of ExperimentalPsychology MonoroTh Supplement, 1968,7t, (I, Pt. 2).
Potter, M. C., & Levy, Z. I. Recognition memory for a rapid sequenceof pictures. Journal of Experimental Psychology, 1969, 81, 10-15.
Rollins, H. A., i Thibadeou, R.recognition of information1973, 1, 164-168.
33
The effects of auditory shadowing onreceived visually. Memory & Cognition,
Sampson, J. R. Free recall of verbal and non-verbal stimuli. QuarterlyExperimental 1970, 22, 215-221.
Seamon, J. G., & Gazzaniga, M. S. Coding strategies and cerebrallaterality effects. Cognitive Psychology, 1973, 5, 249-256.
Shaffer, W. 0., & Shiffrin, a. M. Rehearsal and storage of visualinformation. Journal of Experimental Psychology, 1972, 92,292-296.
Shepard, R. N. Recognition memory for words, sentences, and pictures.Journal of Verbal LearningLand Verbal Behavior, 1967, 6, 156-163.
Smith, E. E., & Larson, D. E. The verbal loop hypothesis and the effectsof similarity on recognition and communication in adults andchildren. Journal of Verbal Learning and Verbal Behavior, 1970,9, 237-242.
Snodgrass, J. G., Volvovitz, R., 4 Walfish, E. R. Recognition memoryfor words, pictures, and words + pictures. Psychonomic Science,1972, 27, 345-347.
Snodgrass, J. G., Wasser, B., Finkelstein, M., & Goldberg, L. B. Onthe fate of visual and verbal memory codes for pictures and words:Evidence for a dual coding mechanism in recognition memory. Journalof Verbal Learning and Verbal Behavior, 1974, 13, 27-37.
Sperry, R. W. Cerebral organization and behavior. Science, 1961, 133,1749-1756.
Standing, L. Learning 10,000 pictures. Quarterly Journal of ExperimentalPsychology, 1973, 25, 207-222.
Ternes, W., & Yuille, J. C. Words and pictures in an STM task. Journalof Experimental Psychology, :1972, 96, 78-86.
Travers, R. M. W. Research and theo related to audiovisual informationtransmission. INV. e amazoo, gan: Western's CampusBookstore, 1167.
Tversky, B. Encoding processes in recognition and recall. CognitivePsychology, 1973, 5, 275-287.
Wicker, F. W. On the locus of picture-word differences in paired-associatelearning. Journal of Verbal Learning and Verbal Behavior 1970,9, 52-57.
34
APPENDIX A
Word Pools for Experiments 1, 2, and 3
Concrete words
accordian clock harp revolver
alligator coin horse snake
ambulance cottage lake streamer
ankle diamond lawn strawberry
arm door lemon sugar
automobile dress leopard tablebaby dove lip toast
barrel fireplace lobster tower
beaver flask meat truck
bird flower microscope trumpet
bottle fork mosquito umbrella
boy fox mountain waterbutter frog nail whale
caterpillar grass potato wine
cigar hammer refrigerator yacht
Abstract words
ability deceit idea perception
advice democracy intellect perjury
agreement disparity interest quality
attitude distraction irony satire
attribute duty jeopardy soul
belief economy knowledge spirit
blandness effort malice suppression
blasphemy ego mastery tendency
capacity essence memory theory
chance exclusion method thought
clemency famility mind truth
concept fallacy moral unreality
crisis fate necessity upkeep
criterion hindrance opinion virtue
custom hypothesis opportunity welfare
35
APPENDIX B
Adjectives Used as Shadowing Material in Experiment 1
High-imagery list
afraid full rich
big golden round
black green soft
blue happy straight
bright hard strong
broken heavy sweet
clean hot tall
clear large thin
cold little warm
cool narrow white
dark old wild
dead poor yellow
deep pretty young
fresh red
Low-imagery listiv
able fixed really
actual former recent
both here simple
chance his slight
civil last special
common main their
easy more that
entire much this
every my true
equal nice useful
false only vain
farther perhaps wrong
few proper
first real
Ss
APPENDIX C
Recognition Scores for Subjects in Experiment 1.
Shadowing
Pictures Words
1
36
Nom-Shadowing
Pictures
His
Si
S2
S3
S4
s5
36
S7
S8
S9
SIO
S11
S12
S13
S14
S15
S16
10
10
10
8
8
8
10
10
10
7
10
8
10
10
10
9
-deLoS
148
10
10
10
10
10
10
8
9
10
10
10
9
10
9
9
10
154
C
9
8
9
7
9
7
9
7
9
7
10
9
8
8
7
8
131
A His
10
8
9
6
8
7
5
8
9
7
7
5
8
8
4
6
115
10
10
10
10
10
10
8
10
8
10
10
10
10
10
10
10
LoS C
Words
A
10
10
10
9
10
10
10
8
5
10
10
9
10
10
10
9
9
9
9
7
1U
9
8
S
5
6
9
7
4
10
6
8
7
9
8
8
9
8
5
9
6
8
8
6
8
8
8
7
156 150 1 121 122
NOTE: !US pictures are the group designated as high in similarityand LoS pictures are those designated as low its similarity."C "words are from the concrete word pool shown in Appendix A,and "A" words are from the abstract .ord pool. The maximumscore for any subject in any cell was 10,
37
APPENDIX D
Analysis of Variance Summary Table for Experiment 1
Source of Variation SS011. df MS
Subject' (S) 87.98 15 5.87
Shadrang Condition (A) .02 1 .02 .00
A x S 79.23 15 5.28
Stimulus Type (B) 221.27 1 221.27 94.87**
B x S 34.99 15 2.33
A x B .77 1 .77 .85
Ax8xS 13.48 15 .90
**p < .01
38
APPENDIX E
Concepts Presented for "Animal, Vegetable, Mineral" Classification
Task Used in Experiment 2
monkey rhubarb petunia
goose lava pecan
daffodil turquoise rock
emerald grass lead
banana squash salt
squirrel sunflower wheat
otter tiger tulip
mica giraff broccoli
chicken goat rhinoceros
peach artichoke sheep
zinnia coal opossum
carbon granite spinach
copper lime jade
stone leopard grape
porpoise bean toad
chinchilla butterfly geode
crystal tin cow
plum corn sand
coyote iron cherry
pansy orange sparrow
opal zebra orchid
hyacinth camel slate
daisy clay lion
penguin mouse garnet
39
APPENDIX F
Recognition Scores for Subjects in Experiment 2
SO
Traysorder
Auditorybtyre order
Adj.cP W
PosedP W
A-V-MeP W
TotalsP W
1 3 1 2 ADJ-AVM-PRO 12 12 16 12 16 12 44 36
2 3 2 1 AVM-PRO-ADJ 13 13 13 12 14 11 40 36
3 1 2 3 PRO-AVM-ADJ 13 11 15 7 15 9 43 27
4 3 1 2 AVM-ADJ-PRO 16 8 6 9 14 8 32 25
5 3 2 1 PRO-ADJ-AVM 16 16 14 12 16 13 46 41
6 2 1 3 ADJ-PRO-AVM 16 14 14 7 13 6 43 27
7 2 3 1 AVM-PRO-ADJ 16 8 14 9 15 14 45 31
8 1 3 2 ADJ-PRO-AVM 16 12 14 8 12 11 42 31
9 2 1 3 PRO-ADJ-AVM 16 12 16 12 14 11 46 35
10 3 1 2 PRO-AVM-ADJ 15 9 14 8 13 10 42 27
11 1 3 2 AVM-ADJ-PRO 14 9 16 13 14 10 44 32
12 1 2 3 ADJ-AVM PRO 15 11 15 7 16 13 46 31
Total 1178 135 167 116.172 128 517 379
a. As indicated in the description of stimulus materials, threeequivalent sets were selected. These were arbitrarily numbered1, 2, and 3. Since there were 12 Ss and 6 possible orders, twoSs were randomly assigned to each possible order.
b. Since there were 12 Ss and 6 possible orders of the 3 auditorytypes of tasks, two Ss were randomly assigned to each possibleorder.
c. This was an auditory shadowing task wherein S shadowed both high andlow imagery adjectives presented at one second intervals.
d. This was an auditory shadowing task wherein S shadowed a proseselection from the United States Constituti6E.
0, This was an auditory classification task wherein S heard nounspresented one every two seconds which were eitheranimal,vegetable, or mineral and wherein S was to respond to each auditoryword with the class it belonged to:
40
APPENDIX G
Analysis of Variance Summary Table for Experiment 2
Source of Variation SS df MS
Subjects (S) 82.78 11 7.53
Type of Shadowing (A) 18.86 2 9.43 2.05
A x S 101.14 22 4.60
Stimulus Type (5) 264.50 1 264.50 95.39**
B x S 30.50 11 2.77
A x B 1.58 2 .79 .19
A x B x S 90.42 22 4.11
**p < .01
41
APPENDIX
Recognition Scores for Subjects in Experiment 31
ShadowPicture' Word
Non-ShadowPicture Word
Total
S1 9 4 8 9 30
S2 9 7 9 9 34
S3 10 0 10 3 23
S4 10 5 10 8 33
S5 9 3 10 5 27
S6 9 2 10 10 31
S7 10 5 10 6 31
S8 9 6 10 4 29
S9 10 5 10 8 33
S10 10 6 10 6 32
Sli 9 5 10 6 30
S12 9 6 10 / 7 32
S13 9 5 JD 10 34
S14 10 7 9 6 32
S15 8 3 10 6 27
S16 9 1 8 7 25
Total 149 70 154 110
1The scores given represent the number correctly identified as"seen before." The maximum cell score possible by a single Sis 10.
42
APPENDIX I
Analysis of Variance Summary Table for Experiment 3
Source of Variation SS df MS
Subjects (S) 39.11 15 2.61
Shadowing Condition (A) 31.64 1 31.64 17.19**
A x S 27.61 15 1.84
Stimulus Type (B) 236.39 1 236.39 82.73**
B x S 42.86 15 2.86
A x B 19.14 1 19.14 9.87**
AxlixS 29.10 15 1.94
**p < .01
43
APPLNDIX J
Average Reported Confidence Levels in Experiment 31
Shadow
Picture Word
Non-ShadowPicture Word
SI 2.1 1.1 2.7 2.5
S2 2.4 2.3 2.9 2.7
53 2.4 .6 3.0 1,7
S4 2.7 2.2 3.0 2.8
SS 2.5 .3 3.0 1.1
S6 2.5 .7 2.9 1.9
S7 2.9 1.9 3.0 2.2
S8 2.4 1.0 2.9 1.5
S9 3,0 2.0 3.0 2.b
SlO 2.7 .9 2.7 1.2
SIl 2.7 .1) 3.0 1.8
S12 2.7 1.7 2.3 .9
S13 2.3 1.6 3.0 2.4
S14 2.5 1.3 2.6 2.2
S15 2.4 1.2 3.0 1.9
S16 2.5 1.1 2.8 2.3
1
Each S was asked to rate each of his responses as to confidence
level.- In scoring, arbitrary values were assigned to these
confidence levels as follows: 'very sure' ti 3, "moderately sure"
a 2, "somewhat unsure" 1, and "just a guess" 0. Then, the
average confidence rating liy each S for each cell was calculated.
44
APPENDIX K
Analysis of Variance Summary Table for Average
Reported Confidence Levels in Experiment 3
Source of Variation SS df MS
...111
Subjects (S) 525.44 15 35.03
Shadowing Condition (A) 420.25 1 420.25 28.82**
A x S 218.75 15 14.58
Stimulus Type (B) 1849.00 1 1849.00 66.34**
B x S 418.00 15 27.87
A x B 60.06 1 60.06 14.79**
A x 8 x S 60.94 15 4.06
**p < .01
45
APPENDIX L
ConceptsIused in Experiment 4
airplane bread drum kiteapple broom duck owlapron bus elephant peararrow car eye pieball cat farmer pigballoon chair fire radiobarn clock fish shipbasket coat flag shoebear corn hand .squirrelbed cow house treebell cup key wagonbook doll king window
1Each concept was represented in both word form and as a simpleline drawing representation.
46
APPENDIX M
Recall Scores for Subjects in Experiment 41
WidowPicture Word
Non-ShadowPicture Word
Si 7 4. 7 6
S2 5 3 7 7
S3 3 1 5 7
S4 3 2 8 7
SS 4 3 9 7
S6 6 5 a 9 11
S7 6 6 6 10
S8 2 3 9 5
S9 5 3 6 7
S10 6 3 7 5
Sli 5 5 10 10
S12 7 3 6 10
S13 3 2 10 5
S14 2 2 7 7
S15 1 2 8 5
Slb 5 J 4 7 6
Total 75 51 121 115
'The scores given represent the number of concepts correctlyrecalled. The maximum cell score possible by a single S
is 12.
47
APPENDIX N
Analysis of Variance Summary Table for Experiment 4
Source of Variation SS df MS F
Subjects (S) 76.93 15 5.12
Shadowing Condition (A) 189.06 1 189.06 96.33**
A x S 29.43 15 1.96
Stimulus Type (B) 14.06 1 14.06 7.16*
B x S 29.43 15 1.96
A x B 5.06 1 5.06 1.63
AxBxS 46.43 15 3.09
*p**p
< .05
< .01