Psychology and AgingDo Judgments of Learning Modify Older Adults’ ActualLearning?Sarah K. Tauber and Amber E. WitherbyOnline First Publication, July 1, 2019. http://dx.doi.org/10.1037/pag0000376

CITATIONTauber, S. K., & Witherby, A. E. (2019, July 1). Do Judgments of Learning Modify Older Adults’Actual Learning?. Psychology and Aging. Advance online publication.http://dx.doi.org/10.1037/pag0000376

Do Judgments of Learning Modify Older Adults’ Actual Learning?

Sarah K. Tauber and Amber E. WitherbyTexas Christian University

Judgments of learning (JOLs) can improve younger adults’ associative learning of related information.One theoretical explanation for this finding is that JOLs strengthen the relationship between the cue andtarget words of a related word pair. This cue-strengthening hypothesis is particularly relevant for olderadults because learning interventions that enhance associations between items typically benefit theirlearning. Thus, we investigated the degree to which JOLs have a direct influence on older adults’learning. To do so, older and younger adults studied a list of related word pairs (Experiments 1 and 2)or weakly related word pairs (Experiments 3, 4, and 5). Half of the participants made a JOL for each pairand half did not. After a filled 3-min retention interval, participants took a cued-recall test. In allexperiments, older adults’ memory performance was not impacted by making JOLs. By contrast, youngeradults who made JOLs recalled significantly more than those who did not. JOLs may not have modifiedolder adults’ learning because of age-related deficits in processing that limited the degree to which JOLsstrengthened cue-target relationships. It is also possible that JOLs encourage attentional reorienting,which older adults do not benefit from because they are already engaged with the materials. An importantdirection for future work will be to explore these possibilities, as well as others.

Keywords: aging, judgments of learning, associative learning, metamemory

Metacognition refers to thoughts about one’s cognition (Dunlo-sky & Metcalfe, 2009; Flavell, 1971), and metacognitive monitor-ing it is commonly measured with judgments of learning (JOLs;for reviews see, Rhodes, 2016; Tauber & Witherby, 2016). JOLsare made by predicting the likelihood of recalling each item on afuture test. Recent work with younger adults has demonstrated thatJOLs can alter their learning (e.g., Mitchum, Kelley, & Fox, 2016;Soderstrom, Clark, Halamish, & Bjork, 2015). Critically, it is notyet known if JOLs modify older adults’ learning. Our primary goalwas to investigate this issue, which is important for several keyreasons. It has obvious implications for older adults’ learning andfor increasing understanding of cognitive aging generally. As well,it is important from an empirical perspective. If JOLs directlyimpact older adults’ learning, then conclusions drawn about age-related effects on memory in the context of making JOLs may beinaccurate or incomplete. As important, exploring the impact of

JOLs on older adults’ learning is relevant for metacognitive theory,specifically the cue-strengthening hypothesis. It is equally impor-tant for key perspectives in cognitive aging including processingand production deficit hypotheses. We begin first by illustratingthe direct JOL effect on younger adults’ learning followed bydiscussion of metacognitive theory and theoretical perspectives ofcognitive aging.

As an example of a direct effect on younger adults’ learning,Soderstrom et al. (2015) had younger adults study word pairs thatwere related (e.g., herb - spice) or unrelated (e.g., net - plant). Halfof the younger adults made a JOL for each word pair and half didnot. After a 3-min retention interval they took a cued-recall test.Performance for unrelated word pairs did not differ between theJOL and no-JOL groups. By contrast, recall for related word pairswas superior for younger adults who made JOLs relative to thosewho did not (cf. Janes, Rivers, & Dunlosky, 2018). The directeffect of JOLs on younger adults’ short-term learning is typicallystrong (e.g., pooled d � .71, obtained from a continuously cumu-lating meta-analysis) and has been extended to a 2-day retentioninterval (pooled d � .66; Witherby & Tauber, 2017).

According to the cue-strengthening hypothesis, making JOLsstrengthens the existing relationship between related items, whichincreases memory for them (Soderstrom et al., 2015). Consider, forinstance, a younger adult studying a related word pair (e.g., doctor -nurse). If he or she makes a JOL after studying the pair, the associ-ation between the cue (doctor) and target (nurse) is more likely to bestrengthened relative to if no JOL is made. By contrast, unrelated pairs(e.g., doctor - biscuit) by definition do not have an existing associa-tion. As such, the cue-target relationship is unlikely to be strengthenedby making JOLs. If later memory tests rely on cue-target associations,then JOLs should increase recall performance for related information,but not for unrelated information.

Sarah K. Tauber and Amber E. Witherby, Department of Psychology,Texas Christian University.

This research was supported by a James S. McDonnell Foundation 21stCentury Science Initiative in Understanding Human Cognition Collabora-tive Activity Grant. Portions of this research were presented at the 58thannual meeting of the Psychonomic Society in 2017 and at the 15th annualmeeting of the Southwestern Psychological Association in 2017. The rawdata for all experiments have been uploaded to the Open Science Frame-work and can be accessed at https://osf.io/x9zaw/. The authors have noconflicts of interest to report.

Correspondence concerning this article should be addressed to Sarah K.Tauber, Department of Psychology, Texas Christian University, TCU Box298920, 2800 South University Drive, Fort Worth, TX 76129. E-mail:uma.tauber@tcu.edu

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

Psychology and Aging© 2019 American Psychological Association 2019, Vol. 1, No. 999, 0000882-7974/19/$12.00 http://dx.doi.org/10.1037/pag0000376

1

This cue-strengthening hypothesis has implications for olderadults’ learning. For example, interventions focused on formingassociations between cues and targets (e.g., instructions to makemeaningful links between words and to create a sentence connect-ing them together) improve older adults’ memory on associativelearning tests (Naveh-Benjamin, Brav, & Levy, 2007). Thus, meth-ods that enhance associations between items during study arelikely to be beneficial for older adults’ memory, which accordingto the cue-strengthening hypothesis, JOLs should do. However, thecue-strengthening hypothesis is limited because the mechanismsthat produce stronger cue-target relationships are not well de-scribed. In other words, it is not clear why JOLs may strengthen therelationship between related information. We consider one possi-bility that that is relevant for older adults.

When studying word pairs in anticipation of a future test,participants likely adopt a variety of learning strategies. Indeed,Mitchum et al. (2016) found that when probed with common studystrategies, younger adults reported using rote rehearsal, sentenceconstruction, and imagery, as well as considering the relationshipbetween cue and target words. When learning word pairs in thecontext of making JOLs, participants may adopt similar strategies,but may also be more likely to self-reflect on their learning.Specifically, when making a JOL, participants are asked to reflectupon their learning of each item and consider the likelihood ofremembering it in the future. If considering this information inresponse to JOL probes differs from what participants normally doduring study, then JOLs should directly impact learning (cf. Eric-sson & Simon, 1980). That is, relative to minimal self-reflectionabout the status of one’s learning, increased self-reflection viaJOLs may strengthen associations between related cues and targetsbecause it may increase attention to and processing of those items.Even so, JOLs are made rapidly and without a great deal of effort,so it is reasonable to suspect that any additional attention orprocessing that results from self-reflection during learning is suf-ficient to improve learning of related information (that is easy tolearn) but not for unrelated information (that is more challengingto learn).

If JOLs modify processing during learning, it may be useful toconsider them in the context of learning strategies. Making JOLsis not commonly viewed as a strategy to use during learning.Instead, as noted earlier, JOLs are most often used to measuremonitoring during learning. Even so, JOLs may encourage aself-reflective learning strategy. As such, aging theory on strategyuse during associative learning can be used to generate predictionsabout whether JOLs will impact older adults’ memory. Accordingto the processing deficit hypothesis of associative learning, olderadults are less successful at using learning strategies than areyounger adults, even when effective strategies are produced (e.g.,Naveh-Benjamin et al., 2007; Naveh-Benjamin, Craik, Guez, &Kreuger, 2005; Shaw & Craik, 1989). Relative to younger adults,processing limitations experienced by older adults may constrainthe degree to which JOLs strengthen cue-target associations. This,in turn would effectively reduce (or eliminate) any direct impact ofJOLs on older adults’ learning when it is evaluated with tests thatare dependent upon cue-target associations. Thus, from this per-spective, JOLs may not influence older adults’ learning because ofdeficits in effective strategy use.

Older adults can also experience impairments in strategy pro-duction. That is, according to the production deficit hypothesis of

associative learning, older adults have difficulty generating effec-tive learning strategies relative to younger adults (e.g., Craik &Byrd, 1982; Naveh-Benjamin et al., 2007; Perlmutter & Mitchell,1982). To address this possibility, in the present context, wepartially controlled for strategy production by requiring older andyounger adults to report a JOL for each item. If older adultsexperience a production deficit, then those who are required tomake JOLs should have superior memory relative to those withoutexplicit instructions to do so. In other words, JOLs should enhanceolder adults’ learning. Further, this effect should be larger on olderadults’ learning relative to younger adults’ learning. Even so, olderadults do not always demonstrate production deficits (e.g., Dun-losky & Hertzog, 1998), and production and processing deficits arenot mutually exclusive.

Experiment 1

In Experiment 1, older and younger adults studied a series ofrelated word pairs. We selected related word pairs becauseexperimenter-paced research with younger adults has consistentlydemonstrated that JOLs improve learning with these materials(e.g., Janes et al., 2018; Soderstrom et al., 2015; Witherby &Tauber, 2017, for a review, see Double, Birney, & Walker, 2018).By contrast, in some instances JOLs improve younger adults’learning of unrelated pairs (e.g., Arbuckle & Cuddy, 1969; Dough-erty, Scheck, Nelson, & Narens, 2005), or single words (Zech-meister & Shaughnessy, 1980), but it is not always observed(Mitchum et al., 2016; Soderstrom et al., 2015; Tauber & Rhodes,2012), or replicated (Dougherty, Robey, & Buttaccio, 2018).

Half of the participants in each age group made a JOL for eachpair and half did not. Next, participants had a 3-min retentioninterval and took a cued-recall test. Consistent with the cue-strengthening hypothesis, we expected younger adults in the JOLgroup to outperform younger adults in the no-JOL group. Predic-tions for older adults are less clear and may depend on the degreeto which production and processing deficits are in evidence. Ac-cording to the production deficit hypothesis, older adults’ learningshould be enhanced by making JOLs when learning related wordpairs. However, older adults may also experience processing def-icits, which would limit the impact of JOLs on their learning.

Method

Design and participants. A 2 (Age Group: younger, older) �2 (Judgment Group: JOL, no-JOL) between-participants designwas used. A power analysis (G�Power; Faul, Erdfelder, Lang, &Buchner, 2007) using an effect size of d � .71 (pooled effect ofJOL vs. no-JOL groups after a 3-min retention interval fromWitherby & Tauber, 2017), power at .9, and an � level of .05estimated that 35 participants would be needed per group. Weincreased the target sample size (i.e., approximately five partici-pants per group) to increase the likelihood of detecting a possibleinteraction between age group and judgment group. Thus, 163participants were recruited: 83 older adults and 80 younger adults.One older adult opted to stop participating half way through theexperiment, and four data files were lost because of experimentererror. As such, data from 78 older adults were used in all analyses.Older adults were randomly assigned to a JOL group (n � 38) orno-JOL group (n � 40). Likewise, younger adults were randomlyassigned to a JOL group (n � 40) or no-JOL group (n � 40).

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

2 TAUBER AND WITHERBY

For all reported experiments, older adults were recruited from anexisting database or from ads placed in local newspapers. All olderadults were screened for (a) currently taking prescription medica-tions or herbal treatments to improve their memory, (b) havingparticipated in memory skills training, and (c) having a diagnosisthat may impact their memory (e.g., dementia, schizophrenia, braintumor, Parkinson’s disease, etc.). Older adults were each compen-sated with a $10 gift card to a local store. Younger adults wererecruited from the participant pool at Texas Christian Universityand each received course credit. All participants in all experiments

were treated in accordance with ethical guidelines and the researchwas approved by an Institutional Review Board.

Descriptive statistics of demographic variables and assessmentsof cognitive functioning can be found in Table 1 (for more infor-mation, see online supplement at https://osf.io/x9zaw/). For allexperiments, education level significantly differed between agegroups, Fs � 85.3, ps � .001. In Experiment 1 (and in allsubsequent experiments), age, gender, and ethnicity did not differbetween judgment groups, and judgment group did not interactwith age group, Fs � 1.60, �2s � 6.96, ps � .14. To evaluate basic

Table 1Descriptive Statistics for Assessments of Cognitive Functioning, Demographic Characteristics, and Recall Performance for All Groupsin Experiments 1–5

MeasureDetailed JOL groups

M (SD)Min. JOL groups

M (SD)No JOL groups

M (SD) 1 2 3 4 5 6

Experiment 11. Pattern 33.6 (8.8) — 33.4 (7.8) 1.002. Letter 20.6 (5.2) — 21.0 (5.1) .67�� 1.003. Vocab 15.6 (7.8) — 16.6 (8.0) �.35��� �.19� 1.004. Age 46.6 (27.3) — 47.6 (27.3) �.60��� �.44��� .69��� 1.005. Gender .81 (.39) — .80 (.40) .01 .03 �.09 �.23�� 1.006. Education 14.7 (2.5) — 15.0 (2.9) �.45��� �.26�� .62��� .68��� �.27��� 1.007. Recall 85.4 (11.8) — 80.9 (14.2) .13 .06 .06 �.16 .005 �.06

Experiment 21. Pattern 33.6 (8.4) — 32.7 (9.4) 1.002. Letter 21.2 (4.8) — 20.2 (5.6) .63��� 1.003. Vocab 16.4 (5.8) — 16.2 (6.3) �.12 .02 1.004. Age 45.2 (25.7) — 46.5 (27.1) �.73��� �.57��� .27��� 1.005. Gender .76 (.43) — .78 (.42) .19� .07 �.02 �.14 1.006. Education 14.4 (3.7) — 14.5 (3.7) �.32��� �.28��� .20�� .47��� �.29��� 1.007. Recall 83.5 (12.9) — 78.2 (15.4) .21�� .18� .10 �.16� .12 .009

Experiment 31. Pattern 32.8 (7.5) — 32.5 (8.6) 1.002. Letter 20.8 (4.6)† — 22.3 (4.8) .69��� 1.003. Vocab 14.5 (6.4) — 13.5 (6.8) �.23�� �.22�� 1.004. Age 45.8 (26.9) — 45.6 (25.7) �.68��� �.53��� .56��� 1.005. Gender .64 (.48) — .61 (.49) .06 .06 �.13 �.10 1.006. Education 14.1 (3.6) — 14.9 (3.0) �.28��� �.24�� .41��� .49��� �.16� 1.007. Recall 76.0 (13.3) — 68.3 (20.7) �.06 �.07 .18� .05 .09 .05

Experiment 41. Pattern 31.4 (10.3) 30.9 (11.0) 29.7 (10.0) 1.002. Letter 19.8 (5.0) 20.1 (5.0) 20.0 (5.1) .74��� 1.003. Vocab 14.9 (7.2) 14.1 (7.4) 13.3 (5.7) �.28��� �.30��� 1.004. Age 45.2 (27.1) 46.1 (28.0) 44.8 (25.9) �.76��� �.73��� .54��� 1.005. Gender .72 (.45) .75 (.44) .67 (.47) .21�� .19�� �.12� �.30��� 1.006. Education 14.5 (3.3) 14.1 (3.9) 14.6 (3.1) �.45��� �.41��� .54��� .61��� �.27��� 1.007. Recall 77.2 (15.7) 77.1 (16.4) 70.5 (17.7) .22��� .28��� .05 �.25��� .25��� �.07

Experiment 51. Pattern — 30.9 (9.9) 29.6 (9.4) 1.002. Letter — 19.7 (5.5) 19.6 (5.0) .75��� 1.003. Vocab — 13.3 (6.2) 14.0 (7.0) �.34��� �.26��� 1.004. Age — 45.8 (27.5) 44.2 (26.9) �.76��� �.70��� .52��� 1.005. Gender — .66 (.48) .70 (.46) .18� .15� �.12 �.18� 1.006. Education — 14.7 (2.9) 14.5 (2.8) �.56��� �.50��� .49��� .75��� �.24��� 1.007. Recall — 67.0 (19.7) 63.2 (19.1) .12 .18� .26��� �.10 .16� �.09

Note. JOL � judgements of learning; Detailed JOL groups � JOL groups with detailed instructions; Min. JOL groups � JOL groups with minimalinstructions; No JOL groups � groups that did not make JOLs; Pattern � the number of items correct on the pattern comparison test; Letter � the numberof items correct on the letter comparison test; Vocab � the number of items correct on the vocabulary test adjusting for guessing (for details, see onlinesupplement at https://osf.io/x9zaw/). SDs of the mean are in parentheses. All variables were continuous except for gender (0 � men, 1 � women).† Significant difference between judgment groups, p � .05. � p � .05. �� p � .01. ��� p � .001.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

3JUDGMENTS OF LEARNING, OLDER ADULTS, AND LEARNING

cognitive functioning, participants completed pattern and lettercomparison tests (Salthouse, 1993) and a vocabulary test (modifiedversion of the advanced vocabulary Test I—v-4, Ekstrom, French,Harman, & Dermen, 1976). Of most interest, in Experiment 1,assessments of cognitive functioning did not differ between thejudgment groups and judgment group did not interact with agegroup, Fs � 1.75, ps � .19 (see Table 1).

Materials and procedure. A 60-item list of word pairs wasused (taken from Witherby & Tauber, 2017). The list consisted ofrelated word pairs (e.g., kiss - hug) from the Nelson, McEvoy, andSchreiber (2004) free-association norms (associated relatednessM � .33, SD � .07). Cue and target words did not differ in length(t � 1), frequency, t(59) � 1.68, p � .10, or the number ofsyllables, t(59) � 1.5, p � .15.

Participants completed the task individually with a researcher inthe room during the entire experiment who read instructions andanswered questions. Participants were told that they would bestudying word pairs presented one-at-a-time, and their task was tolearn each word pair such that they would be able to recall thesecond word of the pair (e.g., hug) when given the first (e.g., kiss)on a later memory test. They were also informed that the wordpairs would be presented for 10 s each, and they would study 60total. To familiarize participants in the JOL groups with the JOLprocedure, they studied and made JOLs for two practice wordpairs. Participants in the no-JOL groups studied the same practiceword pairs. The practice word pairs did not appear on the study listor memory test.

The order of word pairs was randomized anew per participant,and each word pair was presented for 10 s. Participants in theno-JOL groups studied the word pairs for the full 10 s. Participantsin the JOL groups were prompted to make a JOL after 5 s of thestudy time had elapsed. They then had the remaining 5 s to maketheir JOL, during which the word pair remained on the screen. Thisprocedure was used to equate exposure time to each word pairbetween the JOL and no-JOL groups (cf. Soderstrom et al., 2015).Participants were instructed that their goal was to make theirjudgments as accurate as possible (cf. Witherby & Tauber, 2017).They were instructed to consider anything about the word pair thatwould make them likely to recall it. They were given the followingexample to illustrate,

For example, if you see the pair “circus - clown” you may have avery distinct memory of a time you saw a clown at a circus and,thus, may give it a high judgment. On the other hand, if you see thepair “chalk - board,” there may be nothing that jumps out to youas being very memorable. Therefore, you might give this pair a lowjudgment. The key when you make your judgments is to be sure tothoroughly evaluate the likelihood that you will recall the secondword of each pair after 10 min.

Participants orally reported their JOLs to a researcher by re-sponding to the prompt, “Likelihood of recall?” using a 0–100%scale. A .5 s inter-stimulus-interval was used between trials. Par-ticipants in all groups had a 3-min retention interval during whichthey listed states that make up the United States. Next, participantscompleted a cued-recall test, on which they were presented withthe first word of the pair and asked to type the second word of thatpair. The cued-recall test was self-paced and presentation orderwas randomized anew per participant. Finally, participants com-pleted the cognitive battery.

Results and Discussion

Analyses of cued-recall performance were of most interest andare reported for each experiment. Analyses of JOL magnitude andresolution for all experiments can be found in online supplementaldocuments (located at https://osf.io/x9zaw/). Hierarchical linearmodels (HLM) with random participant effects were used in allexperiments (cf. Middlebrooks & Castel, 2018; Middlebrooks,Murayama, & Castel, 2016; Murayama, Sakaki, Yan, & Smith,2014). HLM analyses were conducted because word pairs werenested within-participant and HLM models can account for within-participant variance and nonindependence of data. STATA statis-tical software was used to conduct all HLM analyses (StataCorp,2009).

In Experiment 1 (and all subsequent experiments), typos, spell-ing errors, and minor adjustments to words that did not changetheir meaning (e.g., pluralization) were scored as correct. A logis-tic HLM analysis was conducted that included age group (0 �older adult, 1 � younger adult), judgment group (0 � no-JOL, 1 �JOL), and the interaction between age group and judgment groupto predict participants’ recall performance. In addition, to controlfor sample differences between older and younger adults, weincluded the number of years of education as a covariate in thisanalysis and for analyses in subsequent experiments.

The main effects of age group (older adults, M � 81.64, SE �1.46; younger adults, M � 84.19, SE � 1.62), judgment group, andeducation were not significant, ts � 1. As evident from Figure 1,there was a significant interaction between age group and judg-ment group, b � .78 (SE � .33), t � 2.37, p � .018, 95%confidence interval (CI) [.13, 1.43]. Follow-up analyses revealedthat recall did not differ for older adults based on whether theywere in the JOL or the no-JOL group, t � 1. By contrast, youngeradults in the JOL group recalled significantly more word pairs thandid younger adults in the no-JOL group, b � .74 (SE � .25), t �3.01, p � .003, 95% CI [.26, 1.23]. In summary, JOLs did notmodify older adults’ learning. By contrast, younger adults learningwas improved by making JOLs (cf. Soderstrom et al., 2015;Witherby & Tauber, 2017).

Experiment 2

In Experiment 1, we used an oral reporting procedure for JOLs,which was important for ensuring that participants understood the

Figure 1. The mean percentage correctly recalled by older and youngeradults in the no-JOL (judgements of learning) and JOL groups in Exper-iment 1. Error bars represent 1 SEM.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

4 TAUBER AND WITHERBY

JOL instructions and made a JOL for each pair. Even so, this is notstandard protocol in the metamemory field. Further, this procedureintroduced a social confound between the groups. Participants inthe JOL groups were required to interact with a research assistantthroughout the study and JOL phase, whereas participants in theno-JOL groups were not. Whereas prior research has establishedthat this social confound has relatively no impact on the directeffect of JOLs for younger adults (Witherby & Tauber, 2017), thishas only been demonstrated in one publication. Moreover, it isunclear how the social confound might impact older adults’ learn-ing and memory.

Older adults typically have lower memory self-efficacy than doyounger adults, which means that they believe that their memoryis poorer relative to that of younger adults (or to when they wereyounger). Thus, for older adults in the JOL group, the socialinteraction in Experiment 1 may have produced some anxiety thatmay have distracted them from focusing on JOLs and learning theword pairs. Alternatively, older adults may have experienced so-cial support by interacting with the researcher, which would likelycontribute to their high cued-recall performance in Experiment 1(cf. Hertzog, Sinclair, & Dunlosky, 2010). A primary goal forExperiment 2 was to use a more common JOL procedure thateliminated the social confound by having participants in the JOLgroups enter their own JOLs.

Method

Design and participants. A 2 (Age Group: younger, older) �2 (Judgment Group: JOL, no-JOL) between-participants designwas used. Participants from Experiment 1 were excluded fromparticipating in Experiment 2. Using the power analysis fromExperiment 1, 160 participants were recruited: 80 older adults and80 younger adults. Older and younger adults were randomlyassigned to JOL groups or no-JOL groups (n � 40 in each).Descriptive statistics for demographic characteristics and as-sessments of cognitive functioning can be found in Table 1 (seealso, the online supplement at https://osf.io/x9zaw/). Assess-ments of cognitive functioning did not differ between the judg-ment groups and judgment group did not interact with agegroup, Fs � 2.29, ps � .13.

Materials and procedure. The materials were identical tothose used in Experiment 1. A few key modifications were madeto the procedure used in Experiment 1. Although a researcher waspresent in the room for the JOL and no-JOL groups and for theentire experiment (as in Experiment 1), he or she only readinstructions and answered questions that arose. Participants com-pleted the study (and JOL) phase on their own. Participants in theJOL groups typed their judgments into the computer and wereprovided with the same prompt and scale as used in Experiment 1.All other aspects of the procedure were identical to that of Exper-iment 1.

Results and Discussion

The analysis was identical to that of Experiment 1. The maineffects of age group (older adults, M � 78.33, SE � 1.43; youngeradults, M � 83.29, SE � 1.75), judgment group, and educationwere not significant, ts � 1.02. As evident from Figure 2, therewas a significant interaction between age group and judgment

group, b � .73 (SE � .36), t � 2.02, p � .043, 95% CI [.02, 1.45].Follow-up analyses revealed that recall did not differ for olderadults based on whether they were in the JOL group or the no-JOLgroup, t � 1. By contrast, younger adults in the JOL group recalledsignificantly more word pairs than did younger adults in theno-JOL group, b � .81 (SE � .33), t � 2.43, p � .015, 95% CI[.16, 1.46].

Outcomes from Experiment 2 replicated key outcomes estab-lished in Experiment 1. Specifically, younger adults’ learning wasenhanced by making JOLs relative to not making them, but olderadults’ learning was not. Thus, the social confound evident inExperiment 1 cannot explain the lack of a direct effect of JOLs onolder adults’ learning (cf. Witherby & Tauber, 2017).

Experiment 3

In Experiment 3, we explored the impact of the to-be-learnedmaterials on the degree to which older adults’ learning will beinfluenced by making JOLs. Older and younger adults in Experi-ment 3 learned an entirely new list of word pairs to ensure thateffects found in Experiments 1 and 2 were not bound to oneparticular list of items. Further, in Experiments 1 and 2, partici-pants learned related word pairs because direct JOL effects havebeen most consistent with them. Even so, younger and older adultsperformed well on the cued-recall tests. For example, older adults’cued-recall performance ranged from 78–82% correct, which issurprising (cf. Hertzog et al., 2010). One possibility is that the taskwas not sufficiently challenging and older adults performed at (orclose to) a functional ceiling. If so, it would be challenging todetect a direct effect of JOLs on older adults’ learning. To explorethis possibility, in Experiment 3 we used more challenging to-be-learned materials (i.e., weakly related word pairs). If older adults’memory performance in Experiments 1 and 2 was at a functionalceiling, then with a more challenging task in Experiment 3, evi-dence of JOLs modifying older adults’ learning may arise.

Method

Design and participants. A 2 (Age Group: younger, older) �2 (Judgment Group: JOL, no-JOL) between-participants designwas used. Participants from all prior experiments were excludedfrom participating in Experiment 3. Based on the power analysis

Figure 2. The mean percentage correctly recalled by older and youngeradults in the no-JOL (judgements of learning) and JOL groups in Exper-iment 2. Error bars represent 1 SEM.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

5JUDGMENTS OF LEARNING, OLDER ADULTS, AND LEARNING

from the previous experiments, 160 participants were recruited: 80older adults and 80 younger adults. One older adult did notcomplete the final test. Thus, data from 79 older adults are in-cluded in all analyses. Older adults were randomly assigned to aJOL group (n � 39) or no-JOL group (n � 40). Likewise, youngeradults were randomly assigned to a JOL or no-JOL group (n � 40in each). Descriptive statistics for demographic characteristics andassessments of cognitive functioning are located in Table 1 (see also,the online supplement at https://osf.io/x9zaw/). Regarding assess-ments of cognitive functioning, judgment group differences arose onthe letter comparison test, which indicated that participants in theno-JOL groups outperformed participants in the JOL groups, F(1,155) � 4.27, p � .04, �p

2 � .03. To preview, controlling forperformance on the letter comparison test did not alter recalloutcomes. Performance on the pattern comparison test and vocab-ulary test did not differ between the judgment groups, Fs � 2.46,ps � .12, and judgment group did not interact with age group forany measure, Fs � 1.15, ps � .28.

Materials and procedure. A new 60-item list of word pairswas used. Word pairs were created from the Nelson et al. (2004)norms, and weakly related pairs (e.g., dairy - cow) were selected(associated relatedness M � .15, SD � .03). Cue and target wordsdid not differ in length, t(59) � 1.43, p � .16, frequency, t(59) �1.93, p � .06, or the number of syllables (t � 1). The procedurewas identical to that of Experiment 2.

Results and Discussion

The analysis was identical to that of Experiments 1 and 2. Themain effect of age group was not significant (older adults, M �71.02, SE � 2.13; younger adults, M � 73.27, SE � 1.84),b � �.45 (SE � .24), t � 1.89, p � .06, 95% CI [�.93, .02], norwas the main effect of judgment group, t � 1. The main effect ofeducation was significant, b � .09 (SE � .04), t � 2.24, p � .025,95% CI [.01, .16]. Follow-up analyses revealed that education wasa significant predictor of recall for younger adults, b � .17 (SE �.09), t � 1.93, p � .054, 95% CI [�.003, .34], but not for olderadults, b � .06 (SE � .04), t � 1.51, p � .13, 95% CI [�.02, .14].More important, as evident from Figure 3, there was a significantinteraction between age group and judgment group, b � 1.20(SE � .30), t � 3.92, p � .001, 95% CI [.60, 1.79]. Follow-upanalyses revealed that recall did not differ for older adults in the

JOL group relative to the no-JOL group, t � 1. By contrast,younger adults in the JOL group recalled significantly more wordpairs than did younger adults in the no-JOL group, b � 1.02 (SE �.21), t � 4.79, p � .001, 95% CI [.60, 1.44].

Given that performance on the letter comparison task differedbetween the JOL and no-JOL groups, we conducted an additionalanalysis to ensure that the key effects maintained while controllingfor this variable. The critical two-way interaction maintained, b �1.19 (SE � .31), t � 3.90, p � .001, 95% CI [.59, 1.79]. That is,recall was superior for younger adults in the JOL group relative tothe no-JOL group, b � 1.02 (SE � .22), t � 4.73, p � .001, 95%CI [.60, 1.45], and did not differ for older adults in the JOL andno-JOL groups, t � 1.

In Experiment 3, older adults’ memory was not affected bymaking JOLs. By contrast, younger adults who made JOLs re-called more word pairs than did younger adults who did not. Theseoutcomes replicated those found in Experiments 1 and 2 and, thus,indicate that they were not bound to one specific list of word pairs.Instead, they were observed with related word pairs (Experiments1 and 2) and weakly related word pairs (Experiment 3) that weremore challenging to remember.

Experiment 4

In Experiments 1–3, participants who made JOLs were givendetailed instructions about how to do so. For example, participantswere instructed to give high JOLs to word pairs for which adistinct memory came to mind and to give low JOLs to items forwhich no memory came to mind. Thus, it is possible that the taskinstructions increased the likelihood that participants in the JOLgroups would engage in interactive imagery during study relativeto participants in the no-JOL groups. If so, the memory benefitexperienced by younger adults in the JOL groups would be due tousing an imagery strategy rather than to making JOLs per se. Thetask instructions may have similarly encouraged older adults toengage in interactive imagery, but older adults may have been lesseffective at doing so. In Experiment 4, we explored the impact oftask instructions on older and younger adults’ memory by addingnew JOL groups who received minimal instructions about how tomake their JOLs.

Method

Design and participants. A 2 (Age Group: younger, older) �3 (Judgment Group: JOL with detailed instructions, JOL withminimal instructions, no-JOL) between-participants design wasused. Participants from all prior experiments were excluded fromparticipating in Experiment 4. To estimate sample size, a poweranalysis was conducted using an effect size of �p

2 � .09 (interac-tion between judgment group and age group evident from theanalysis of variance [ANOVA] conducted with data from Exper-iment 3), power at .9, and an � level of .05. This analysis estimatedthat 44 participants per group would be needed to detect theinteraction. Thus, 270 participants were recruited: 134 older adultsand 136 younger adults. Older and younger adults were randomlyassigned to group (n � 45 older adults and 45 younger adults in theJOL with detailed instructions groups, n � 45 older adults and 46younger adults in the JOL with minimal instructions groups, andn � 44 older adults and 45 younger adults in the no-JOL

Figure 3. The mean percentage correctly recalled by older and youngeradults in the no-JOL (judgements of learning) and JOL groups in Exper-iment 3. Error bars represent 1 SEM.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

6 TAUBER AND WITHERBY

groups). Descriptive statistics for demographic characteristicsand assessments of cognitive functioning are located in Table 1(see also, the online supplement at https://osf.io/x9zaw/). As-sessments of cognitive functioning did not differ between thejudgment groups, and judgment group did not interact with agegroup, Fs � 1.92, ps � .15.

Materials and procedure. The materials were identical tothose used in Experiment 3. The procedure for the no-JOL groupswas identical to the procedure for the no-JOL groups from Exper-iments 1, 2, and 3. The procedure for the JOL groups with detailedinstructions was identical to the procedure for the JOL groups fromExperiments 2 and 3. Participants in the JOL groups with minimalinstructions were given the same information about the word pairs,studying them, the JOL scale, and the test relative to participantsin the JOL groups with detailed instructions. However, participantsin the JOL groups with minimal instructions were only given thefollowing instructions for making JOLs,

After you study a pair for 5 s, we would like you to make a judgmentof how likely you are to remember the second word of the pair whengiven the first. To do so, a box will appear on the screen while theword pair is still visible, and you will make your judgment anticipat-ing a memory test that will occur in about 10 min.

All other aspects of the procedure were identical to that of Exper-iments 2 and 3.

Results and Discussion

A logistic HLM analysis was conducted predicting recall as afunction of age group (0 � older adult, 1 � younger adult) andjudgment group. For judgment group, two dummy codes werecreated. A detailed JOL code was created to compare the JOLgroup with detailed instructions to the no-JOL group (no-JOL � 0,JOL detailed � 1) and a minimal JOL code was created tocompare the JOL group with minimal instructions to the no-JOLgroup (no-JOL � 0, JOL minimal � 1).

The main effects of age group (older adults, M � 71.24, SE �1.47; younger adults, M � 78.58, SE � 1.37) and education werenot significant, ts � 1.02. Of interest, the interaction between agegroup and the detailed JOL code (no-JOL vs. JOL detailed) wassignificant, b � 1.12 (SE � .29), t � 3.85, p � .001, 95% CI [.55,1.69], as was the interaction between age group and the minimalJOL code (no-JOL vs. JOL minimal), b � .72 (SE � .29), t � 2.46,p � .014, 95% CI [.15, 1.29]. As evident from Figure 4, theinteractions revealed that recall did not differ for older adults in theno-JOL group and in the JOL group with detailed instructions (t �1), or for older adults in the no-JOL group and in the JOL groupwith minimal instructions (t � 1). In contrast, younger adults in theJOL group with detailed instructions recalled more word pairs thandid those in the no-JOL group, b � .96 (SE � .20), t � 4.80, p �.001, 95% CI [.57, 1.36], and younger adults in the JOL group withminimal instructions recalled more word pairs than did those in theno-JOL group, b � .78 (SE � .20), t � 3.84, p � .001, 95% CI[.38, 1.17].

In a second logistic HLM analysis, the JOL group with detailedinstructions was used as the reference group to compare recall forparticipants in the two JOL groups (i.e., JOL group with detailedinstructions and the JOL group with minimal instructions). Recalldid not differ between the JOL groups, t � 1.01, and the interac-

tion between age group and the JOL group code (JOL detailed vs.JOL minimal) was not significant, b � �.40 (SE � .29), t � 1.36,p � .18, 95% CI [�.97, .18]. Thus, recall did not differ for the JOLgroup with minimal instructions and the JOL group with detailedinstructions for either age group.

In summary, older adults’ memory was not impacted by makingJOLs, which was true regardless of whether they received detailedor minimal JOL instructions. By contrast, younger adults whomade JOLs (regardless of the JOL instructions) recalled more thandid younger adults who did not. Thus, the direct JOL effect onyounger adults’ learning is attributable to the process associatedwith making JOLs, rather than to a byproduct of the task instruc-tions.

Experiment 5

Even though older adults recalled numerically fewer word pairsrelative to younger adults in Experiments 1–4, these effects did notreach conventional levels of statistical significance. This is sur-prising, given that age-related deficits in paired-associated learningare well-established (though, the decrement is typically small forrelated pairs, e.g., Naveh-Benjamin, 2000; Naveh-Benjamin et al.,2005; Naveh-Benjamin, Hussain, Guez, & Bar-On, 2003). Onepossibility that persists is that the learning task is insufficientlydemanding, and older adults’ learning is functioning at full capac-ity. If so, JOLs would have minimal impact on older adults’ recallperformance. As one way to address this issue, in Experiments 3and 4 participants studied weakly related word pairs instead of themore strongly related pairs used in Experiments 1 and 2. Even so,older adults’ learning was not influenced by making JOLs.

In Experiment 5, we adopted a more challenging procedurerelative to those used in Experiments 1–4 by reducing participants’study time by half. By doing so, participants in the JOL groups hadless time to make their JOLs. Given this time constraint, an oralreport procedure for JOLs was used in Experiment 5 to minimizeconcerns that older adults may not have enough time to enter theirJOLs. Participants’ JOLs were tape-recorded, so no interactionwas required with a researcher. As such, the social confoundpresent in Experiment 1 was not reintroduced in Experiment 5.Participants studied the weakly related word pairs from Experi-ments 3 and 4, and those in the JOL groups were given onlyminimal instructions for making JOLs. If JOLs did not modify

Figure 4. The mean percentage correctly recalled by older and youngeradults in the no-JOL (judgements of learning) groups, JOL groups withdetailed instructions, and JOL groups with minimal instruction in Experi-ment 4. Error bars represent 1 SEM.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

7JUDGMENTS OF LEARNING, OLDER ADULTS, AND LEARNING

older adults’ learning in Experiments 1–4 because the task was tooeasy, then the procedure used in Experiment 5 should yield thehighest likelihood of observing a direct effect of JOLs on olderadults’ learning.

Method

Design and participants. A 2 (Age Group: younger, older) �2 (Judgment Group: JOL, no-JOL) between-participants designwas used. Participants from all prior experiments were excludedfrom participating in Experiment 5. Based on the power analysisfrom Experiment 4, 182 participants were recruited: 91 olderadults and 91 younger adults. Older adults were randomly assignedto a JOL group (n � 45) or no-JOL group (n � 46). Likewise,younger adults were randomly assigned to a JOL (n � 45) orno-JOL group (n � 46). Descriptive statistics for demographiccharacteristics and assessments of cognitive functioning are lo-cated in Table 1 (see also, the online supplement at https://osf.io/x9zaw/). Assessments of cognitive functioning did not differ be-tween the judgment groups, and judgment group did not interactwith age group, Fs � 2.97, ps � .09.

Materials and procedure. The materials were identical tothose of Experiments 3 and 4. Each word pair was presented for 5s. Participants in the no-JOL groups studied the word pairs for thefull 5 s. Participants in the JOL groups were prompted to make aJOL after 2.5 s of the study time had elapsed and used theremaining 2.5 s to orally make their JOL with the word pair on thescreen. To increase the likelihood that participants would make aJOL for each pair, participants in the JOL group heard a tone after2.5 s of study had elapsed with the instruction that the tonesignaled to orally report their JOL. They continued studying theword pair for the remaining time.

Participants in the JOL groups were given similar instructions tothe minimal JOL groups in Experiment 4. Minor additions weremade to the JOL instructions to inform participants about the toneand how to report their JOLs. Specifically, participants were giventhe following instructions,

After you study a pair for 2.5 s, we would like you to make a judgmentof how likely you are to remember the second word of the pair whengiven the first. To do so, a tone will sound while the word pair is stillvisible, and you will make your judgment verbally (i.e., say it outloud) anticipating a memory test that will occur in about 10 min.

Instructions about the JOL scale were then provided, whichwere identical to instructions used in all prior experiments. Then,participants were given the following instructions,

Your judgments will be recorded by the tape recorder that is sitting onthe desk. When you are making your judgments, it is critical that youspeak loud enough so that the tape recorder can hear you. Moreover,ensure that you enunciate each number clearly. Before finishing theexperiment, a researcher will check the recording to ensure that yourresponses have been recorded.

To equate participants’ auditory experiences, participants in theno-JOL groups also heard a tone after 2.5 s of study had elapsed.These participants were instructed that the tone signaled that 2.5 sremained to study the word pair. Otherwise, the instructions for theno-JOL groups were identical to those of the no-JOL groups in theprior experiments. For all groups, a researcher read instructions to

participants (also provided on a computer screen), answered ques-tions, and then left the room before any to-be-learned materialswere presented. The retention interval and cued-recall test wereidentical to those used in the prior experiments.

Results and Discussion

The analysis was identical to that of Experiments 1, 2, and 3.Nonsignificant main effects of age group (older adults, M � 64.52,SE � 1.80, younger adults, M � 66.96, SE � 2.03), b � �.30(SE � .25), t � 1.22, p � .22, 95% CI [�.79, .18], education (t �1), and judgment group (t � 1), were qualified by a significantinteraction between age group and judgment group, b � .66 (SE �.28), t � 2.38, p � .02, 95% CI [.12, 1.19]. As evident from Figure5, older adults’ recall did not differ between the JOL and no-JOLgroups, t � 1. By contrast, younger adults in the JOL grouprecalled significantly more than did younger adults in the no-JOLgroup, b � .59 (SE � .21), t � 2.80, p � .005, 95% CI [.18, 1.00].

Even though cued-recall performance was lower in Experiment5 relative to the prior experiments, outcomes replicated thoseestablished in Experiments 1–4. Older adults’ memory was notimpacted by making JOLs, but younger adults’ memory was.Specifically, younger adults who made JOLs recalled more thandid younger adults who did not.

General Discussion

The reported experiments were the first to investigate the directimpact of JOLs on older adults’ learning, relative to their impacton younger adults’ learning. There was no evidence of JOLsmodifying older adults’ learning. That is, older adults’ learningwas not influenced by making JOLs regardless of whether JOLswere reported orally (Experiments 1 and 5) or typed (Experiments2, 3, and 4), and whether older adults studied related wordpairs (Experiments 1 and 2) or weakly related word pairs (Exper-iments 3, 4, and 5). As well, JOLs did not impact older adults’learning regardless of whether detailed JOL instructions (Experi-ments 1, 2, 3, and 4) or minimal JOL instructions (Experiments 4and 5) were provided, and whether a 10 s study rate (Experiments1, 2, 3, and 4) or a 5 s study rate was allowed (Experiment 5).

In stark contrast with older adults’ learning, JOLs improvedyounger adults’ learning, which was evident in all experiments.Moreover, this effect was substantial. Younger adults who made

Figure 5. The mean percentage correctly recalled by older and youngeradults in the no-JOL (judgements of learning) and JOL groups in Exper-iment 5. Error bars represent 1 SEM.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

8 TAUBER AND WITHERBY

JOLs recalled on average approximately 10–18% more than didthose who did not make JOLs (Experiment 1, 9.5%, d � .70;Experiment 2, 9.8%, d � .66, Experiment 3, 18%, d � 1.07,Experiment 4, detailed instruction Group 15.7%, d � 1.07, mini-mal instruction Group 15.7%, d � .78, Experiment 5, 11.1%, d �.60). These outcomes replicate the direct JOL effects previouslyfound when younger adults learn related information (e.g., Soder-strom et al., 2015; Witherby & Tauber, 2017), and are consistentwith the more general contention that JOLs can modify youngeradults’ learning (e.g., Janes et al., 2018; Mitchum et al., 2016).

The lack of a direct JOL effect on older adults’ learning issurprising considering that in previous research, procedures thatstrengthen associations between cues and targets typically improveolder adults’ learning when they must retrieve those associations toperform well on tests (cf. Naveh-Benjamin et al., 2007). Accordingto the cue-strengthening hypothesis, JOLs enhance learning ofrelated information because they reinforce the association betweenword pairs. Thus, older adults who made JOLs should have re-called more relative to older adults who did not. In what follows,we consider four explanations for the lack of a direct effect ofJOLs on older adults’ learning. Specifically, we consider twohypotheses related to older adults’ learning (processing and pro-duction deficits), and two attention perspectives (attentional reori-enting and divided attention).

Perspectives on Older Adults’ Learning

In contrast with studying without making JOLs, making JOLsmay modify processing of pairs during study because participantsare asked to reflect on the status of their learning and predict futurememory performance. Considering such information for each itemduring learning may deviate from how participants study withoutresponding to JOL probes, which may impact subsequent learning(cf. Ericsson & Simon, 1980). However, in contrast with youngeradults, older adults have limited cognitive resources and, accord-ing to the processing deficit hypothesis, they are less effective atusing learning strategies (e.g., Naveh-Benjamin et al., 2005; Shaw& Craik, 1989). Thus, self-reflection as a learning strategy mayhave been ineffective for older adults. As such, JOLs may not havestrengthened the cue-target association to the same degree (or foras many items) for older adults relative to younger adults’ JOLs.This may account for the absent JOL effect on older adults’memory performance.

Relative to a processing deficit, it is less likely JOLs did notmodify older adults’ learning because of a production deficit.According to the production deficit hypothesis, older adults areless successful at generating effective learning strategies than areyounger adults (e.g., Craik & Byrd, 1982; Naveh-Benjamin et al.,2007; Perlmutter & Mitchell, 1982). In the current experiments, wepartially controlled for strategy production in the JOL groupsbecause participants were given instructions to make JOLs andwere required to do so. By contrast, participants in the no-JOLgroups needed to produce JOLs on their own to benefit from them.If, relative to younger adults, older adults were less likely toproduce monitoring judgments (such as JOLs) in the no-JOLgroups, then a direct effect of JOLs should have been larger onolder adults’ learning than on younger adults’ learning. This age-related effect was not obtained in any experiment. However,strategy reports were not obtained from participants, so it is un-

clear which learning strategies older and younger adults used.Some prior work with younger adults suggests that self-reportedencoding strategies are unaffected by JOLs (Mitchum et al., 2016,Experiment 1), but it is unknown whether the same is true of olderadults. Investigating the degree to which older and younger adults’strategies differ because of making JOLs and whether such differ-ences contribute to JOLs as memory modifiers is an importantdirection for future research.

Perspectives on Attention

Another mechanism that may impact processing during studywhen JOLs are made (relative to when they are not made) isattentional reorienting. Consider that in the current experiments,participants in the no-JOL groups used the entire study time tolearn each word pair. It is reasonable to suspect that their attentionto the task may have waned and mind wandering may haveincreased as the presentation duration progressed. By contrast, inthe JOL groups, time was broken up by first studying the pair andthen receiving a prompt to make a JOL. Thus, participants mayhave reengaged with each word pair as a result of making a JOL,which would likely reduce mind wandering and increase attentionto the learning task. If so, the additional attention may be relativelyminor, and may be most likely to benefit learning of relatedmaterials because these associations are typically easy to form andretain. In contrast, learning more challenging materials such asunrelated word pairs, definitions for key terms, or lengthy textsmay not benefit from reengagement due to making a JOL.

More important, in a related literature, refreshing during work-ing memory tasks (i.e., bringing information back into the focus ofattention) is predictive of episodic memory performance (e.g.,Loaiza & McCabe, 2012, 2013). However, older adults typicallybenefit less from refreshing opportunities than do younger adults(Johnson, Reeder, Raye, & Mitchell, 2002; Loaiza & McCabe,2013). As well, increased mind wandering may be more evidentfor younger adults than for older adults (e.g., Giambra, 1993;Giambra, 2000; Krawietz, Tamplin, & Radvansky, 2012). In ourexperience, older adults often come to the lab eager to participateand voice intentions to do their best. Younger adults do nottypically demonstrate the same enthusiasm or goals. Thus, a directeffect of JOLs may not have been observed on older adults’learning because they were actively attending to word pairs moreso than were younger adults, and consequently any reorienting thatJOLs may have caused did not impact them. Investigating thevalidity of this reorienting hypothesis will be an important direc-tion for future research.

As a final consideration, it is possible that older adults’ learningwas not influenced by JOLs because making JOLs concurrentlywith acquisition creates a divided attention task only for olderadults, or that is more challenging for older adults than for youngeradults (Mitchum et al., 2016, cf. Stine-Morrow, Shake, Miles, &Noh, 2006). This dual-task hypothesis may account for the nega-tive effect of JOLs on younger adults’ learning that is sometimesobserved for learning of unrelated word pairs. Moreover, someresearchers have found that divided attention impacts older adults’performance more negatively than it does younger adults’ perfor-mance (e.g., Park, Smith, Dudley, & Lafronza, 1989; Salthouse,Rogan, & Prill, 1984). Thus, if making JOLs while learning relatedword pairs creates a challenging divided attention task, then we

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

9JUDGMENTS OF LEARNING, OLDER ADULTS, AND LEARNING

would predict that older adults would not benefit from makingthem. That said, the strongest prediction from the dual-task hy-pothesis would be that completing both tasks (i.e., learning wordpairs while making JOLs) should harm learning relative to com-pleting the learning task by itself. As such, JOLs should have adirect and negative effect on both older and younger adults’learning, and it should be larger for older adults. Given that wefound no evidence of a negative effect of JOLs on learning, ifmaking JOLs does produce a divided attention task for olderadults, it may only have a relatively minor impact on their learn-ing.

Implications and Recommendations for FutureResearch

It is important to note that the cue-strengthening hypothesis issomewhat limited because direct evidence of strengthened rela-tionships between related cues and targets is currently absent(leading to circularity). As well, it is unclear which mechanism(s)produce strengthened associations between related items. As aresult, it is not well understood why direct JOL effects on learningare observed in some circumstances and not others. Take forexample, the direct effect of JOLs on learning of weakly relatedword pairs. Consistent with younger adult outcomes reported inExperiments 3–5, JOLs have been shown to improve youngeradults’ memory of weakly related information (Arbuckle &Cuddy, 1969; Dougherty et al., 2005). However, in other instances,JOLs do not impact memory for weakly related pairs (Kelemen &Weaver, 1997; Soderstrom et al., 2015). It is possible that differ-ences in methodology and stimuli (e.g., degree of associationbetween weakly related words) may contribute to inconsistenteffects. An important direction for future research will be tosystematically explore the impact of JOLs on older and youngeradults’ learning of various kinds of information with the goal toinform theory about mechanisms that account for direct JOLeffects.

Given that the current experiments demonstrated no evidence ofa direct effect of JOLs on older adults’ learning, one mightconclude that metacognitive judgments are unimportant for olderadults’ learning. However, we caution against this conclusion. Itmay be that older adults’ learning will be influenced by JOLs (orother metacognitive judgments) in other contexts. In at least oneother case, metacognitive judgments can directly impact olderadults’ learning. Specifically, remember or know judgments, adifferent kind of monitoring judgment, can enhance older adults’associative learning (Naveh-Benjamin & Kilb, 2012). As well,according to contemporary metacognitive theory, JOLs (as well asother metacognitive judgments) can also influence learning indi-rectly. JOLs can impact people’s study decisions, which can sub-sequently impact learning (e.g., Dunlosky & Hertzog, 1997; Finn,2008; Thiede, 1999).

It is also important to consider that even though older adultshave well-established deficits in episodic memory compared withyounger adults, such deficits did not arise in the reported experi-ments. What could account for this? There are exceptions toage-related declines in episodic memory (e.g., Castel, 2005;Flores, Hargis, McGillivray, Friedman, & Castel, 2017), and insome instances, older adults perform as well as do younger adultson episodic memory tests (e.g., Tauber, Dunlosky, Urry, & Opitz,

2017; Tomaszczyk & Fernandes, 2013). Further, the type of to-be-learned materials can impact the degree to which age-relatedassociative deficits are observed. In Experiments 1 and 2, youngerand older adults studied related word pairs (e.g., lightning - thun-der, kiss - hug, alarm - clock) and in Experiments 3–5, they studiedweakly related word pairs (e.g., dairy - cow, cheese - mouse,patient - hospital). Age-related differences in memory perfor-mance are often smaller for related pairs relative to other kinds ofmaterial (e.g., Naveh-Benjamin, 2000; Naveh-Benjamin et al.,2003, 2005). It is possible that older adults relied on their semanticmemory to remember a portion of the to-be-learned word pairs,which would account for the small difference in cued-recall be-tween younger and older adults. A fruitful direction for futureresearch will be to explore the impact of JOLs on older andyounger adults’ learning of both unrelated and related pairs ofwords because age-related deficits in learning may appear forunrelated pairs.

A limitation of the reported experiments is that older adults wererecruited from the community whereas younger adults were re-cruited from the university participant pool. Such recruiting pro-cedures are common; however, they may contribute to differencesbetween samples. One possibility is that the samples of olderadults obtained were high-functioning relative to the larger popu-lation of older adults. To partially control for this, we includedself-reported years of education in HLM analyses of memoryperformance in all experiments. Even so, we cannot eliminate thepossibility that older adults in the reported experiments werehigh-functioning relative to the larger population of older adults.An important direction for future research will be to adopt anindividual difference approach and explore the degree to whichmeasures of executive functioning moderate the effect of JOLs onolder adults’ learning. Relative to older adults who are high-functioning, older adults who are low-functioning may be morelikely to benefit from making JOLs.

Conclusions

Making JOLs did not modify older adults’ learning. By com-parison, younger adults’ learning was significantly enhanced bymaking JOLs. The lack of a direct effect of JOLs on older adults’learning may be attributable to processing deficits that constrainedthe degree to which JOLs strengthen cue-target associations. It isalso possible that JOLs encourage reorienting to pairs during studyfor younger adults, but that older adults do not need this attentionalshift and, thus, do not benefit from it. As well, JOLs may create adual-task that is more challenging for older adults than for youngeradults. Additional research is necessary to explore these explana-tions (as well as others) for direct JOL effects on older adults’learning.

References

Arbuckle, T. Y., & Cuddy, L. L. (1969). Discrimination of item strength attime of presentation. Journal of Experimental Psychology, 81, 126–131.http://dx.doi.org/10.1037/h0027455

Castel, A. D. (2005). Memory for grocery prices in younger and olderadults: The role of schematic support. Psychology and Aging, 20, 718–721. http://dx.doi.org/10.1037/0882-7974.20.4.718

Craik, F. I. M., & Byrd, M. (1982). Aging and cognitive deficits: The roleof attentional resources. In F. I. Craik & S. Trehub (Eds.), Aging andcognitive processes (pp. 191–211). New York, NY: Plenum Press.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

10 TAUBER AND WITHERBY

Double, K. S., Birney, D. P., & Walker, S. A. (2018). A meta-analysis andsystematic review of reactivity to judgements of learning. Memory, 26,741–750. http://dx.doi.org/10.1080/09658211.2017.1404111

Dougherty, M. R., Robey, A. M., & Buttaccio, D. (2018). Do metacogni-tive judgments alter memory performance beyond the benefits of re-trieval practice? A comment on and replication attempt of Dougherty,Scheck, Nelson, and Narens (2005). Memory & Cognition, 46, 558–565.http://dx.doi.org/10.3758/s13421-018-0791-y

Dougherty, M. R., Scheck, P., Nelson, T. O., & Narens, L. (2005). Usingthe past to predict the future. Memory & Cognition, 33, 1096–1115.http://dx.doi.org/10.3758/BF03193216

Dunlosky, J., & Hertzog, C. (1997). Older and younger adults use afunctionally identical algorithm to select items for restudy during mul-titrial learning. The Journals of Gerontology: Series B, 52, P178–P186.http://dx.doi.org/10.1093/geronb/52B.4.P178

Dunlosky, J., & Hertzog, C. (1998). Aging and deficits in associativememory: What is the role of strategy production? Psychology and Aging,13, 597–607. http://dx.doi.org/10.1037/0882-7974.13.4.597

Dunlosky, J., & Metcalfe, J. (2009). Metacognition. Thousand Oaks, CA:Sage.

Ekstrom, R. B., French, J. W., Harman, H. H., & Dermen, D. (1976).Manual for kit of factor-referenced cognitive tasks. Princeton, NJ: Ed-ucational Testing Service.

Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psycho-logical Review, 87, 215–251. http://dx.doi.org/10.1037/0033-295X.87.3.215

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G�Power 3: Aflexible statistical power analysis program for the social, behavioral, andbiomedical sciences. Behavior Research Methods, 39, 175–191. http://dx.doi.org/10.3758/BF03193146

Finn, B. (2008). Framing effects on metacognitive monitoring and control.Memory & Cognition, 36, 813–821. http://dx.doi.org/10.3758/MC.36.4.813

Flavell, J. H. (1971). First discussant’s comments: What is memory de-velopment the development of? Human Development, 14, 272–278.http://dx.doi.org/10.1159/000271221

Flores, C. C., Hargis, M. B., McGillivray, S., Friedman, M. C., & Castel,A. D. (2017). Gist-based memory for prices and “better buys” in youngerand older adults. Memory, 25, 565–573. http://dx.doi.org/10.1080/09658211.2016.1197944

Giambra, L. M. (1993). The influence of aging on spontaneous shifts ofattention from external stimuli to the contents of consciousness. Exper-imental Gerontology, 28, 485– 492. http://dx.doi.org/10.1016/0531-5565(93)90073-M

Giambra, L. M. (2000). Frequency and intensity of daydreaming: Agechanges and age differences from late adolescent to the old-old. Imag-ination, Cognition and Personality, 19, 229–267. http://dx.doi.org/10.2190/XN4W-1CRE-B0MH-84XT

Hertzog, C., Sinclair, S. M., & Dunlosky, J. (2010). Age differences in themonitoring of learning: Cross-sectional evidence of spared resolutionacross the adult life span. Developmental Psychology, 46, 939–948.http://dx.doi.org/10.1037/a0019812

Janes, J. L., Rivers, M. L., & Dunlosky, J. (2018). The influence of makingjudgments of learning on memory performance: Positive, negative, orboth? Psychonomic Bulletin & Review, 25, 2356–2364. http://dx.doi.org/10.3758/s13423-018-1463-4

Johnson, M. K., Reeder, J. A., Raye, C. L., & Mitchell, K. J. (2002).Second thoughts versus second looks: An age-related deficit in reflec-tively refreshing just-activated information. Psychological Science, 13,64–67. http://dx.doi.org/10.1111/1467-9280.00411

Kelemen, W. L., & Weaver, C. A., III. (1997). Enhanced metamemory atdelays: Why do judgments of learning improve over time? Journal ofExperimental Psychology: Learning, Memory, and Cognition, 23, 1394–1409. http://dx.doi.org/10.1037/0278-7393.23.6.1394

Krawietz, S. A., Tamplin, A. K., & Radvansky, G. A. (2012). Aging andmind wandering during text comprehension. Psychology and Aging, 27,951–958. http://dx.doi.org/10.1037/a0028831

Loaiza, V. M., & McCabe, D. P. (2012). Temporal-contextual processingin working memory: Evidence from delayed cued recall and delayed freerecall tests. Memory & Cognition, 40, 191–203. http://dx.doi.org/10.3758/s13421-011-0148-2

Loaiza, V. M., & McCabe, D. P. (2013). The influence of aging onattentional refreshing and articulatory rehearsal during working memoryon later episodic memory performance. Aging, Neuropsychology, andCognition, 20, 471– 493. http://dx.doi.org/10.1080/13825585.2012.738289

Middlebrooks, C. D., & Castel, A. D. (2018). Self-regulated learning ofimportant information under sequential and simultaneous encoding con-ditions. Journal of Experimental Psychology: Learning, Memory, andCognition, 44, 779–792. http://dx.doi.org/10.1037/xlm0000480

Middlebrooks, C. D., Murayama, K., & Castel, A. D. (2016). The value inrushing: Memory and selectivity when short on time. Acta Psychologica,170, 1–9. http://dx.doi.org/10.1016/j.actpsy.2016.06.001

Mitchum, A. L., Kelley, C. M., & Fox, M. C. (2016). When asking thequestion changes the ultimate answer: Metamemory judgments changememory. Journal of Experimental Psychology: General, 145, 200–219.http://dx.doi.org/10.1037/a0039923

Murayama, K., Sakaki, M., Yan, V. X., & Smith, G. M. (2014). Type Ierror inflation in the traditional by-participant analysis to metamemoryaccuracy: A generalized mixed-effects model perspective. Journal ofExperimental Psychology: Learning, Memory, and Cognition, 40, 1287–1306. http://dx.doi.org/10.1037/a0036914

Naveh-Benjamin, M. (2000). Adult age differences in memory perfor-mance: Tests of an associative deficit hypothesis. Journal of Experimen-tal Psychology: Learning, Memory, and Cognition, 26, 1170–1187.http://dx.doi.org/10.1037/0278-7393.26.5.1170

Naveh-Benjamin, M., Brav, T. K., & Levy, O. (2007). The associativememory deficit of older adults: The role of strategy utilization. Psychol-ogy and Aging, 22, 202–208. http://dx.doi.org/10.1037/0882-7974.22.1.202

Naveh-Benjamin, M., Craik, F. I. M., Guez, J., & Kreuger, S. (2005).Divided attention in younger and older adults: Effects of strategy andrelatedness on memory performance and secondary task costs. Journalof Experimental Psychology: Learning, Memory, and Cognition, 31,520–537. http://dx.doi.org/10.1037/0278-7393.31.3.520

Naveh-Benjamin, M., Hussain, Z., Guez, J., & Bar-On, M. (2003). Adultage differences in episodic memory: Further support for an associative-deficit hypothesis. Journal of Experimental Psychology: Learning,Memory, and Cognition, 29, 826–837. http://dx.doi.org/10.1037/0278-7393.29.5.826

Naveh-Benjamin, M., & Kilb, A. (2012). How the measurement of mem-ory processes can affect memory performance: The case of remember/know judgments. Journal of Experimental Psychology: Learning, Mem-ory, and Cognition, 38, 194–203. http://dx.doi.org/10.1037/a0025256

Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004). The Universityof South Florida free association, rhyme, and word fragment norms.Behavior Research Methods, Instruments & Computers, 36, 402–407.http://dx.doi.org/10.3758/BF03195588

Park, D. C., Smith, A. D., Dudley, W. N., & Lafronza, V. N. (1989).Effects of age and a divided attention task presented during encodingand retrieval on memory. Journal of Experimental Psychology: Learn-ing, Memory, and Cognition, 15, 1185–1191. http://dx.doi.org/10.1037/0278-7393.15.6.1185

Perlmutter, M., & Mitchell, D. B. (1982). The appearance and disappear-ance of age differences in adult memory. In F. I. Craik & S. Trehub(Eds.), Aging and cognitive processes (pp. 127–144). New York, NY:Plenum Press. http://dx.doi.org/10.1007/978-1-4684-4178-9_7

Rhodes, M. G. (2016). Judgments of learning: Methods, data, and theory.

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

11JUDGMENTS OF LEARNING, OLDER ADULTS, AND LEARNING

In J. Dunlosky & S. K. Tauber (Eds.), The Oxford handbook ofmetamemory (pp. 65–80). New York, NY: Oxford UP.

Salthouse, T. A. (1993). Speed and knowledge as determinants of adult agedifferences in verbal tasks. Journal of Gerontology, 48, P29–P36. http://dx.doi.org/10.1093/geronj/48.1.P29

Salthouse, T. A., Rogan, J. D., & Prill, K. A. (1984). Division of attention:Age differences on a visually presented memory task. Memory & Cog-nition, 12, 613–620. http://dx.doi.org/10.3758/BF03213350

Shaw, R. J., & Craik, F. I. M. (1989). Age differences in predictions andperformance on a cued recall task. Psychology and Aging, 4, 131–135.http://dx.doi.org/10.1037/0882-7974.4.2.131

Soderstrom, N. C., Clark, C. T., Halamish, V., & Bjork, E. L. (2015).Judgments of learning as memory modifiers. Journal of ExperimentalPsychology: Learning, Memory, and Cognition, 41, 553–558. http://dx.doi.org/10.1037/a0038388

StataCorp. (2009). Stata statistical software: Release 11. College Station,TX: Author.

Stine-Morrow, E. A. L., Shake, M. C., Miles, J. R., & Noh, S. R. (2006).Adult age differences in the effects of goals on self-regulated sentenceprocessing. Psychology and Aging, 21, 790–803. http://dx.doi.org/10.1037/0882-7974.21.4.790

Tauber, S. K., Dunlosky, J., Urry, H. L., & Opitz, P. C. (2017). The effectsof emotion on younger and older adults’ monitoring of learning. Neu-ropsychology and Cognition, 24, 555–574. http://dx.doi.org/10.1080/13825585.2016.1227423

Tauber, S. K., & Rhodes, M. G. (2012). Measuring memory monitoringwith judgements of retention (JORs). Quarterly Journal of ExperimentalPsychology, 65, 1376–1396. http://dx.doi.org/10.1080/17470218.2012.656665

Tauber, S. K., & Witherby, A. E. (2016). Metacognition in older adulthood.In N. A. Pachana (Ed.), Encyclopedia of geropsychology (pp. 1–15).New York, NY: Springer. http://dx.doi.org/10.1007/978-981-287-080-3_215-1

Thiede, K. W. (1999). The importance of monitoring and self-regulationduring multitrial learning. Psychonomic Bulletin & Review, 6, 662–667.http://dx.doi.org/10.3758/BF03212976

Tomaszczyk, J. C., & Fernandes, M. A. (2013). A positivity effect in olderadults’ memorability judgments of pictures. Experimental Aging Re-search, 39, 254–274. http://dx.doi.org/10.1080/0361073X.2013.779178

Witherby, A. E., & Tauber, S. D. (2017). The influence of judgments oflearning on long-term learning and short-term performance. Journal ofApplied Research in Memory & Cognition, 6, 496–503. http://dx.doi.org/10.1016/j.jarmac.2017.08.004

Zechmeister, E. B., & Shaughnessy, J. J. (1980). When you know that youknow and when you think that you know but you don’t. Bulletin of thePsychonomic Society, 15, 41–44. http://dx.doi.org/10.3758/BF03329756

Received July 8, 2018Revision received May 28, 2019

Accepted June 3, 2019 �

Thi

sdo

cum

ent

isco

pyri

ghte

dby

the

Am

eric

anPs

ycho

logi

cal

Ass

ocia

tion

oron

eof

itsal

lied

publ

ishe

rs.

Thi

sar

ticle

isin

tend

edso

lely

for

the

pers

onal

use

ofth

ein

divi

dual

user

and

isno

tto

bedi

ssem

inat

edbr

oadl

y.

12 TAUBER AND WITHERBY