MOTIVATING PERSISTENCE IN THE FACE OF FAILURE: THE IMPACT OF

AN EGO-PROTECTIVE BUFFER ON LEARNING CHOICES AND OUTCOMES

IN A COMPUTER-BASED EDUCATIONAL GAME

A DISSERTATION

SUBMITTED TO THE SCHOOL OF EDUCATION

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Catherine Chase

August 2011

http://creativecommons.org/licenses/by-nc/3.0/us/

This dissertation is online at: http://purl.stanford.edu/yw409rc6957

© 2011 by Catherine Chi Chase. All Rights Reserved.

Re-distributed by Stanford University under license with the author.

This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.

ii

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Daniel Schwartz, Primary Adviser

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Paulo Blikstein

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Carol Dweck

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Deborah Jane Stipek

Approved for the Stanford University Committee on Graduate Studies.

Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.

iii

iv

ABSTRACT

This research begins to explore motivational supports for learning from failure

situations. Failure often presents a valuable learning opportunity, and in fact, many

instructional models have capitalized upon it. However, in order to learn from their

mistakes, students may need motivational scaffolds to protect them from the negative

psychological ramifications of failure.

This work explored the effectiveness of a motivation-based intervention called

an ego-protective buffer (EPB), that was designed to enhance persistence after failure.

An ego-protective buffer (EPB) maintains a stable sense of competence by lessening

the impact of failure on one’s psyche. The specific instantiation of an EPB tested here

was designed to elicit a combination of internal and external attributions for failure.

External attributions protect one’s sense of competence by averting the blame for

failure away from the self. This should discourage one from quitting the task. At the

same time, this ego-protective buffer invites some internal attributions, which

encourages students to take some responsibility for remedying the failure situation.

Based on this theory, we embedded an EPB into the rule structure of a

computer-based genetics game and unleashed it on 143 seventh graders. In the EPB

condition, students were told that winning in the game was a probabilistic outcome,

dependent on a combination of chance and skill on the part of the students. In the

Control condition, students were told that winning in the game was a deterministic

outcome, dependent on students’ skill only. Students played the game during two

class periods. Measures include pre- and posttests, motivational survey measures, and

in-game behaviors.

The EPB did have an effect on learning, but only amongst high-failing

students. High-failing EPB students learned just as much as their low-failing

counterparts. This was not so in the Control condition, where high-failing students

learned far less than their low-failing counterparts. So the high-failing EPB group was

behaving as if they were “buffered” from the effects of failure. We also found

evidence of a possible mechanism behind this learning effect. In the high-failing EPB

condition, students were equally likely to persist after success and failure, while in the

v

Control condition, students were far more likely to persist after success, exhibiting risk

averse behaviors. This difference was more exaggerated in a within-subjects

comparison, contrasting the same individuals in situations of high and low failure.

Finally, persistence after failure was associated with learning across the full sample of

subjects. Regardless of condition or failure rate, students who persisted more after

failure also learned more. This study, together with the author’s related body of work,

provides compelling evidence that the EPB is a viable intervention for encouraging

persistence in the face of failure.

This study also made some headway in pioneering new measures of learning

process and motivational behaviors. In this study, behavioral measures of persistence

after failure were able to predict learning gains. However, no motivational survey

measures could predict the persistence after failure behaviors. This suggests that these

behavioral measures can provide us with some unique functional measures of

motivation that adaptable learning environments could target for intervention.

vi

ACKNOWLEDGEMENTS

This dissertation was the work of many fine people. I would like to thank my

advisor, Daniel Schwartz, for his undying support in every single aspect of this work!

And certainly this research could not have been completed without the help of Doris

Chin and Ilsa Dohmen, who provided feedback in the design of the game, helped to

administer the study, and helped to code some of the data. The game would not have

been possible without Henry Kwong, our trusty programmer, who magically built and

maintained the game, the database, and a data-searching interface. My dissertation

support group (Marily Oppezzo, Karin Chapin, Kathleen O’Connor, Lindsay Oishi,

Maryanna Rogers, and Heidy Maldonado) offered incredible feedback and unfailing

support throughout the entire process. My family and friends (especially Adam

Royalty) were essential in helping me get through this monster of a dissertation.

Finally, I would like to thank Kimi Schmidt and her students, who were gracious

enough to participate in this study!

vii

TABLE OF CONTENTS

Chapter 1 Introduction 1 Chapter 2 Relevant prior research 21 Chapter 3 Methods 34 Chapter 4 Effects of treatment on learning and persistence: A four-

pronged analysis 51

Chapter 5 Behaviors before and after success and failure 78 Chapter 6 Pre-existing individual differences as predictors of in-game

behaviors and learning outcomes 90

Appendix to Chapter 6 114 Chapter 7 Choices of discrete events 120 Appendix to Chapter 7 129 Chapter 8 Discussion 132 Chapter 9 Reflections, implications, and future directions 142 Appendix 158 References 192

viii

LIST OF TABLES

Number Title Page

Table 2.1 Partial correlations with posttest for the Unscaffolded condition, removing variance due to pretest and number of fails

33

Table 3.1 Condition assignment by period and day 43 Table 3.2 Interview questions 44 Table 4.1 Average scores on pretest, posttest, learning gains, and transfer

items 55

Table 4.2 Calculation of learning gain scores 56 Table 4.3 Correlations between learning gains and rates of success- and

fail-abandon, overall and split by condition 59

Table 4.4 Correlations between the rates of success- and fail-abandon and

learning gain scores, broken out by condition and high- and low-failers

65

Table 4.5 Summary of findings from four data analyses, addressing three

research questions 75

Table 5.1 Rates of various types of responses to success and failure, split

by condition and high/low failers 80

Table 5.2 Rates of actions preceding success and failure, split by

condition and high/low failers 83

Table 5.3 Correlations between learning gains and events preceding

success and failure 86

Table 6.1 Reliability analyses for each motivation scale 94 Table 6.2 Average ratings on motivation measures given prior to

treatment 95

Table 6.3 Correlations between motivation measures, prior achievement,

and learning gains 95

ix

Table 6.4 Correlations between motivation measures and persistent behaviors in the game

114

Table 6.5 Correlations between measures of motivation and pre- and post-

treatment learning measures, split by condition 98

Table 6.6 Correlations between motivation measures and persistent

behaviors in the game, split by condition 114

Table 6.7 Average ratings on motivation measures, split by high and low

learners 99

Table 6.8 Correlations between motivation measures and pre and post

learning measures, broken out by high and low failers 115

Table 6.9 Correlations between motivation measures and persistent

behaviors, split by high and low failers 115

Table 6.10 Correlations between motivation measures and learning

measures, split by condition and high/low failers 116

Table 6.11 Correlations between motivation measures and in-game

behaviors, split out by condition and high/low failers 117

Table 6.12 Means and SEs of prior achievement and prior knowledge

measures, split by condition and high/low failers 104

Table 6.13 Correlation matrix relating pre- and post-treatment learning

measures 105

Table 6.14 Correlations between persistence behaviors and pre- and post-

treatment measures of learning 106

Table 6.15 Correlations matrix of pre- and post-treatment learning

measures, split by condition 107

Table 6.16 Regression results testing condition, science grade, and their

interaction 107

Table 6.17 Regression results testing condition, failure rate, and their

interaction 108

Table 6.18 Correlations between learning measures and behaviors

preceding success and failure, split by condition 109

x

Table 6.19 Correlation matrix between pre- and post-learning measures, split by high and low failers

110

Table 6.20 Correlations between persistence behaviors and pre- and post-

treatment learning measures, split by high/low failers 118

Table 6.21 Correlation matrix of pre- and post-learning measures split by

condition and high/low failers 111

Table 6.22 Correlations between pre and post learning measures and

responses to failure, split by condition and high and low failers 119

Table 7.1 Sums and proportions of choices and failures 122 Table 7.2 Correlations between choice types and learning learning gains 129 Table 7.3 Sums and proportions of in-game choices and failures, split by

condition 129

Table 7.4 Correlations between proportions of discrete choices and

learning gains 129

Table 7.5 High and Low failers’ mean sums and proportions of choices 125 Table 7.6 Correlations between proportions of discrete choices and

learning gains, for low and high failers 130

Table 7.7 Proportions of various activities by high/low failers and

condition 130

Table 7.8 Correlations with learning gain score by high/low failers and

condition 131

xi

LIST OF ILLUSTRATIONS

Number Title Page

Figure 1.1 Achievement attributions categorized by locus, stability, and controllability 5

Figure 2.1 Screenshot of Betty’s Brain concept-mapping interface 23 Figure 2.2 Screenshot of the Triple-A Game Show and chat window 24

Figure 2.3 Posttest scores broken out by item type, achievement level, and condition 26

Figure 2.4 Set-up of learning choices for the Scaffolded condition 31 Figure 3.1 Homepage of Mendel’s Galaxy 37 Figure 3.2 Example puzzle 38 Figure 3.3 Hierarchy of game choices 39 Figure 3.4 Failure message screens for EPB and Control conditions 42 Figure 3.5 Sample post-test items 47 Figure 4.1 Rates of fail-abandon and success-abandon by condition 58 Figure 4.2 Histograms of failure rates by condition 61 Figure 4.3 Learning gain scores by condition by high- versus low-failers 62 Figure 4.4 Scatterplots of failure rate by learning gain for each condition 63

Figure 4.5 Abandonment after success and failure by condition and high- versus low-failers 64

Figure 4.6 Learning gains by high/low failure situations by condition 67

Figure 4.7 Rates of abandonment after success and failure split by condition and high/low failure situations 69

Figure 4.8 Learning gain by condition and high/low prior knowledge situations 72

xii

Figure 4.9 Rates of abandonment after success and failure split by condition and high/low prior knowledge situations 73

1

CHAPTER 1: INTRODUCTION

Students encounter many learning failures throughout their school careers.

Whether solving a math problem, coming up with an idea for an essay, or writing a

computer program, students’ first attempts often fail. Learning failures, however

small, are an inevitable part of education.

To learn from these failure situations, students must persist, and they must

persist productively. We consider these two issues separately. The first half of this

chapter explores the motivational mores of why people do or do not persist after

receiving negative evaluations. According to attribution theory, persistence after

failure is strongly influenced by the perceived cause of the failure. While much of the

literature in this space has defined stark dichotomies between attribution types, we

argue that people can assign multiple causes to outcomes, and that this can be a

healthy response to failure. We suggest a possible intervention (the ego-protective

buffer) based on this idea and propose an experiment to test its effect on persistence.

But even when students do persist after failure, they may not engage in the

kind of persistence that is productive for learning. If students perseverate on failing

strategies or select poor learning behaviors, their learning will stagnate and they will

continue to confront failure. The second half of this chapter attempts to characterize

the types of learning behaviors that comprise productive persistence. Models of self-

regulated learning offer some guidance in this effort, however there is little evidence

connecting specific self-regulated learning behaviors to learning outcomes.

Computer-based measures of help-seeking behaviors have made headway in this

regard, but they are limited in scope because the computer exerts a high level of

control over students, which restricts opportunities for self-regulated learning.

Moreover, help-seeking and self-regulated learning have not been studied from the

perspective of failure. To fill this gap, we propose a descriptive analysis to

characterize the types of behavioral moves students make following failure in a

choice-filled environment. We will then attempt to empirically derive which

behaviors are productive by relating specific behaviors to learning outcomes.

2

What is failure?

Failure comes in many shapes and sizes, but it always begins with a goal.

Failure occurs when the goal is not met. The goal can be self-adopted (e.g. I want to

learn Chinese) or assigned (e.g. homework). It can be large (e.g. completing high

school) or small (e.g. getting this question right). And, of course, the goal can be

situated in different contexts like sports or shopping. This dissertation considers

failure in the context of learning and achievement. We further confine the discussion

to “micro” failures that occur at the level of answering a question incorrectly, solving

a problem incorrectly, or failing a low-stakes quiz.

A situation becomes a failure only when one perceives that the goal has not

been met. Both external (e.g. a grade) and internal (e.g. I don’t know how to begin

this problem) feedback can indicate failure. Also, failure is not always black and

white; there are different degrees of failure. Scoring 10 points out of 100 on a quiz is

a far more catastrophic failure than scoring 75 points. This adds issues of relative

standards and expectations to the perception of failure. Some students view a score of

75 as a failure, while others see it as a triumph. This paper simplifies the discussion of

failure to an all-or-nothing outcome and deals largely with failure that is defined by

salient external feedback.

Learning from failure

Failure can provide useful feedback for learning. Failure denotes a problem in

meeting the learning objective. It draws one’s attention to a knowledge gap, a flaw in

understanding, or a computational mistake. Failure can also serve as a call-to-action,

triggering the learner to remedy the error, modify one’s understanding, or revise one’s

work. In the view advanced in this paper, failure can be a valuable learning

opportunity.

Some even use failure as a productive learning tool. For instance, scientists

often make new discoveries when experiments come out differently than predicted. In

a study of a molecular biology lab, Dunbar (1999) found that biologists paid particular

3

attention to evidence that indicated their theories were failing. Asking why these

results had occurred often pushed them to generate novel theories and innovations in

their field. The same principle applies for children learning in school. Japanese math

teachers make students’ problem-solving mistakes a focal part of their instruction

(Stigler & Hiebert, 1999). Likewise, Italian math teachers encourage students to self-

correct their errors rather than give them the answer, as American teachers are wont to

do (Santagata, 2005).

Learning activities can be designed to capitalize on the idea that students learn

from making or analyzing mistakes. For instance, Siegler (2002) found that asking

students to explain both incorrect and correct solutions led to greater rates of transfer

than explaining correct solutions alone. Schwartz & Martin (2004) and Kapur (2008)

have created activities that lead to productive failure, where students generate several

solutions to a complex, open-ended problem. While students rarely generate workable

solutions, they are well prepared to learn the correct solution and its attendant

concepts, when they are later explained to them. Blair (2009) found that students

learn through “implication feedback” where students see the consequences of their

errors and begin to diagnose the misconception in their mathematical problem-solving.

Finally, Vanlehn et al. (2003) found evidence for “impasse-driven learning”, where

students learn more from tutor explanations given after they have reached an impasse

compared to before. This body of work points to the idea that failure can produce

fertile grounds for robust learning. But in order to learn from failure, students must be

able to cope with the failure itself and persist, despite difficulty.

Motivating Persistence after Failure

Failure is almost never enjoyable. The experience of failure is associated with

negative emotion, lowered self-esteem, reduced intrinsic motivation, and lower

expectancies of future success (Covington & Omelich, 1981; Reeve, Olson, & Cole,

1985; Vallerand, Gauvin & Halliwell, 1986). Repeated failure can lead to learned

helplessness and even depression (Bandura, 1997; Peterson, Maier, & Seligman,

1993). A common self-protective response to failure is to quit the task entirely,

4

avoiding the risk of future failure and all its negative ramifications. But quitting the

task impedes learning. People need motivational resources to support persistence after

failure, so they can continue to learn.

Persistence has been explored by several different motivational theories and

constructs, each of which has a different explanation and set of predictions for

persistence in the face of failure. For instance, Bandura’s social cognitive theory

predicts that individuals who are high in self-efficacy are more resilient after failure

because they believe they are capable of performing the tasks necessary for success

(Bandura, 1997). Likewise, theories of intrinsic motivation claim that individuals with

high levels of intrinsic motivation for a given task will persist after failure because

they enjoy the task in its own right (Deci & Ryan, 1985). Achievement goal theories

predict that relative to performance-oriented individuals, individuals who adopt a

mastery goal will be less likely to give up after failure because they are less interested

in “looking good in the eyes of others” and are more interested in learning the material

(Ames & Archer, 1988; Dweck, 1986). While these theories all make valid points, we

contend that the perceived cause of failure is critical in shaping the choice to persist or

quit. Attribution theory provides the most direct theoretical account of the perceived

cause of failure.

Attribution theory and the perceived cause of failure

Attribution theory works on the assumption that individuals are motivated to

master tasks in their environment (Weiner, 1986). To gain control over their

environment and themselves, they try to understand why outcomes occur by

generating attributions, which are hypothesized causes for past events. People tend to

make causal attributions for events that are unexpected, significant, or negative. These

attributions have psychological consequences such as altered expectancies of success,

self-efficacy, and emotions, which in turn affect behaviors like persistence, effort, and

choice.

Attributions can be characterized along three dimensions: locus, stability, and

controllability (Weiner, 1986). The locus of the attribution designates whether the

5

cause of an outcome is internal or external to the person. The stability dimension

refers to whether the cause is permanent or likely to change. Controllability refers to

whether the individual has the power to change the outcome. Each attribution has

characteristics along all three dimensions and can be classified in the 2 x 2 x 2 matrix

shown in Figure 1.1. Most attributions in achievement situations fall into one of four

categories: luck, effort, task difficulty, and ability.

Internal External Controllable Uncontrollable Controllable Uncontrollable

Stable long-term effort

fixed ability teacher bias* task difficulty

Unstable short-term effort, skill, strategy

illness help from others*

luck

Figure 1.1. Achievement attributions categorized by locus, stability, and controllability. These dimensions have been confirmed through both rational and empirical (e.g. factor) analyses (Weiner, 1986). We have italicized the four most common attributions for achievement. *The starred causes are controllable by others (i.e. the teacher) but are not controllable by the person making the attribution. Figure adapted from Weiner, 1979.

The dimensions of an attribution determine persistence in the face of failure.

For instance, if a student believes she earned a bad grade on her physics test because

physics is a hard subject, then she has attributed the poor grade to an external, stable,

and uncontrollable cause. The subject of physics is external to the student, she cannot

control it, and it is not going to change. In this case, the student can do nothing to

improve her grade, so there is no logical recourse other than to give up trying.

However, if instead, she believes her bad grade was due to low short-term effort on

her part, then the attribution is internal, unstable, and controllable. In this situation the

student would be likely to persist by studying harder for the next test.

How do people generate attributions? The attribution an individual forms for a

past event depends on a number of environmental and personal factors (Weiner,

1986). For instance, there are individual differences in attributional styles, such that

some individuals may be more prone to attribute negative outcomes to internal,

uncontrollable, and stable causes (Peterson et al., 1982). Other individual differences

that could affect attributions are prior knowledge and causal schemas and scripts.

6

Environmental factors that include social norms and the specifics of a given situation

can also influence attributions. For instance, if many people in a class perform poorly

on a test, a student is likely to attribute her failure to the difficulty of the test rather

than to her own ability. Given that both individual differences and environmental

factors affect attributions, interventions can target either the person or the situation.

The intervention proposed here uses aspects of the environment to influence students’

perception of failure.

What kinds of attributions will enhance or hinder persistence?

What kinds of attributions for failed outcomes hinder persistence? Most

attribution theorists agree that attributions to uncontrollable causes should be avoided.

In a classic experiment by Seligman & Maier (1967), one group of dogs received

inescapable shocks while another group learned to avoid the shocks by pressing a bar.

Twenty-four hours later, both groups received shocks that could be avoided by

jumping. The bar-pressing group tried many different behaviors to avoid shock and

eventually learned to jump. The other group did not attempt to escape the shock;

many of the dogs lay there passively and accepted it. Learned helplessness occurs

when individuals perceive no relationship between their behavior and the negative

event, rendering the outcome uncontrollable. Learned helplessness can also be

provoked in humans by exposing them to unsolvable problems. In one study, adults

worked on either unsolvable problems, solvable problems, or no problems (Hiroto &

Seligman, 1975). They were then exposed to an uncomfortable loud noise that could

be turned off by the push of a button. Subjects in the unsolvable problem condition

failed to escape the noise, while the other subjects readily pressed the button.

Supposedly, the learned helplessness individuals did not attempt to press the button

because they believed that their behaviors were independent of the negative outcome.

One uncontrollable attribution that is fairly common in achievement situations

but also incredibly maladaptive, involves blaming aptitude or fixed ability. Fixed

ability is stable, uncontrollable, and internal. This means there is nothing the student

can do about it, the situation is not going to change, and the child is stuck with her

7

inborn intelligence. Diener and Dweck (1978) compared helpless and mastery

children’s response to failure at discrimination problems. The helpless children

attributed failure to uncontrollable factors like lack of ability or task difficulty, used

less effectual strategies, and engaged in fewer self-monitoring behaviors. In contrast,

the mastery children made very few attributions and instead made more strategy-

monitoring statements. Diener and Dweck concluded that mastery children were less

interested in dwelling on past failures and more interested in ensuring future mastery

of the task. The helpless children’s attributions to uncontrollable causes, on the other

hand, gave them little incentive to exert effort, which produced undesirable learning

behaviors.

If attributions to uncontrollable causes hinder persistence, which attributions

enhance it? Most attribution theorists agree that effort is the healthiest attribution for

people to make after failure. It diverts the cause of failure away from low ability

while still placing responsibility on the self and encouraging continued work on the

task (Schunk, Pintrich, & Meece, 2008; Weiner, 1979). In other words, effort is an

internal, controllable, and unstable attribution; it contains all the ingredients for

persistence after failure. Dweck (1975) taught helpless children to attribute failure to

effort and compared them to children who only experienced success. The success-

only group’s performance deteriorated after failure, but the helpless children’s

performance either improved or stayed the same. Other interventions that encourage

effort or strategy attributions have enhanced children’s learning and performance as

well (Relich, Debus, & Walker, 1986; Schunk, 1982). Presumably, attributing failure

to effort encourages students to exert greater effort in the future, which often translates

to persistence.

Some scholars argue that the healthiest attribution to make is the most credible

one. Schunk, Pintrich, & Meece (2008) advise teachers to give students realistic

feedback for failure outcomes because students may discount the feedback if it is not

plausible. For instance, if a student does poorly despite exerting tremendous effort,

and the teacher tells the student to try harder, this can lead the student to disregard the

teacher’s feedback altogether. Blumenfeld et al. (1982) also argue for clear and

8

accurate feedback that will allow students to focus on the skills and knowledge they

need to improve. We contend that realistic attributions – which often fall into multiple

categories – can be healthy for learning and persistence.

Blended attributions

Attribution theory has made a significant contribution to our understanding of

motivational responses to success and failure. The dimensions of locus, stability, and

controllability have predictive power, and they aptly classify attributions along three

distinct characteristics. At the same time, the elegance of the framework has led to

interventions that treat categories within a dimension as mutually exclusive when, in

real life, they are not.

One reason attribution categories are not mutually exclusive is that a single

event can have many causes, which even young children know (Leddo, Abelson, &

Gross, 1984). In many surveys that measure attributions, children are asked to rate the

likelihood of several different causes for the same achievement outcome. They often

rate each of these causes as likely or even highly likely (Elig & Frieze, 1979; Frieze &

Bar-Tal, 1980). And of course, most adults and children by the age of twelve believe

that their performance in school is largely a result of both effort and ability (Nicholls,

1990). This evidence suggests that people naturally make multiple attributions for an

outcome, which is sensible given that in reality, all outcomes have multiple causes.

One study of failure attributions found that experts assign both internal and

external causes to failure when working within their domain of expertise (Chase, in

press). In this study, experts in math and English attempted difficult tasks in both

math and English domains. As expected, the experts persisted less and also made less

progress at the out-of-domain task. While working out-of-domain, experts

spontaneously made internal attributions for failure. Prototypical comments include “I

don’t have the skills to do this” and “I’ve never been good at math.” But when

working in-domain, they made a mixture of internal and external attributions for

failure, including “I should know this” and “this poem is horribly written”. These data

9

suggest that people do, under some circumstances, spontaneously attribute failure to

multiple causes simultaneously.

Experts are an interesting case for the study of persistence. They are not only

experts at what they do; they are also motivational experts at what they do. Experts

put in 10,000 hours of effortful, deliberate practice, where they strategically place

themselves in failure situations and learn from them (Bereiter & Scardamalia, 1992;

Chase & Simon, 1973; Ericsson, 2002). If the experts in this study made a

combination of internal and external attributions for failure – then perhaps this is a

productive motivational behavior. Of course, there are probably many reasons why

the experts persisted longer at the task in their domain, including high self-efficacy,

intrinsic interest, and prior knowledge. However, we would like to consider the

possibility that a combination of internal and external attributions also helped these

experts to persist in the face of failure.

Can a combination of internal and external attributions enhance persistence?

Why might a combination of internal and external attributions enhance

persistence? Let us begin by discussing the beneficial effects of external attributions.

Attributing failure to external causes is a common response to failure, and some have

argued that it is serves to protect the self. The locus of attribution is known to affect

self-esteem or self-worth (Weiner, 1986; Covington & Omelich, 1981). Attributing

failure to internal causes tends to lower self-esteem, and people tend to avoid

situations that lower their feelings of self-worth (Rhodewalt et al., 1991). On the other

hand, attributing failure to external causes maintains self-esteem and should decrease

the likelihood that a learner will avoid the failure situation by giving up on the task.

For example, giving “excuses” for negative personal outcomes – the purest form of

which is an external attribution – is beneficial for maintaining self-esteem and positive

affect while reducing anxiety and depression (Snyder & Higgins, 1988; Schlenker,

Pontari, & Christopher, 2001). Snyder and Higgins (1988) argue that external

attributions take the focus off the self and free up attentional resources that can be

used towards the task.

10

Attributing failure to external causes is part of the naturally occurring “self-

serving bias.” The self-serving bias is the tendency to attribute failure externally

while attributing success to internal sources. A recent meta-analysis (Mezulis, et al.,

2004) found that the self-serving bias is widespread and fairly consistent across many

cultures, ages, and various demographics. The self-serving bias has been associated

with increased happiness, less depression, better physical health, and enhanced

problem-solving (Taylor & Brown, 1988). In fact, Allport (1937) called it “nature’s

eldest law” – a natural response to success and failure that helps us maintain a positive

outlook on life.

While external attributions have many advantages for protecting a sense of

self, they do not promote persistence. External attributions take responsibility for

failure away from the learner, making personal strivings unlikely to impact future

success. For instance, some learned helpless individuals will attribute failure to task

difficulty – an external cause over which the learner has no control (Dweck, 1975).

As we have seen, this type of fatalistic attitude can be particularly damaging to

persistence. Internal attributions, on the other hand, place the responsibility for failure

on the learner’s shoulders. Some internal attributions, like effort, can drastically

enhance persistence and performance (Dweck, 1975; Relich, Debus, & Walker, 1986;

Schunk, 1982).

Is there a way to balance internal and external attributions? Ideally, students

would assign some responsibility for a failure to the environment to protect

themselves. At the same time, they would also take responsibility for the failure with

an eye towards effort, so there is a reason for them to take action and persist. These

balanced situations do exist. For instance, when authors receive a “reject-resubmit”

review on a journal submission, they can simultaneously blame the reviewers while

taking responsibility for writing the revision. By placing the blame on an external

source, the author is “freed” from the shackles of failure and all the negative

consequences that come with it. Rather than focus on protecting the self from the

unwelcome ramifications of failure, the author’s mental resources are available to

focus on the task of revision. But at the same time, the author perceives the cause for

11

failure as partially internal and remediable. This spurs persistence in the face of

failure.

It may also be possible to create school environments that support a sweet-spot

of balanced attributions that lead to more effort and persistence. In the next section,

we describe learning conditions that invite students to make a combination of internal

and external attributions for failure, which we call a type of “ego-protective buffer.”

The ego-protective buffer

An ego-protective buffer (EPB) maintains a stable sense of competence by

lessening the impact of failure on one’s psyche. There are many types of ego-

protective buffers. For instance, high self-efficacy could protect one from feeling the

negative ramifications of failure (Bandura, 1997). Attributing failure to effort can act

as an ego-protective buffer by shielding the learner from an attack on her sense of

intelligence (Dweck, 2000).

In this study, we propose that a combination of internal and external

attributions for failure could provide an EPB. External attributions for failure protect

the learner from negative self-thoughts like “I simply don’t have the ability to do this”

or “I’m stupid.” This is important because when learners focus on negative self-

thoughts, their cognitive resources are pulled away from the task at hand, and learning

is much less likely to occur. External attributions can also dissuade the learner from

engaging in self-protective behaviors like quitting the task or selecting tasks that prove

their competencies rather than grow them.

It is important to note that an EPB is not a barrier but a buffer, so it does not

entirely shield the learner from making internal attributions for failure. However,

these internal attributions serve an important function. By placing some of the

responsibility for failure in the learner’s lap, internal attributions can encourage the

learner to take action to fix the failure.

One example of this type of internal-external ego-protective buffer is found in

games. In most games, winning requires an element of luck (an external cause) but

still gives players some internal control over the outcome (using strategies or gaining

12

skills). For instance, black jack players can increase their odds of winning by using

heuristics and strategies, though there is still a huge element of chance involved. So

when players lose a round they can attribute it to bad cards, but they can also take

actions to avoid failure in the future (e.g. “stay” when cards sum to 16 or higher). This

may explain why games can be so addictive, even when people fail.

The implementation of an EPB in this study is a function of the environment

rather than a direct manipulation of students’ attributions. Many attribution-based

interventions have an adult (the teacher or experimenter) make attributions for a

student (Relich, Debus, & Walker, 1986; Schunk, 1982; Schunk, 1983). For instance,

in a study by Schunk (1983), the experimenter occasionally told students “You’ve

been working hard” as they worked on math problems. Rather than directly make the

desired attributions for the child, a situational EPB invites the child to make the

desired attributions herself, by setting up situations where both internal and external

causes for failure seem plausible.

One caveat is that the internal-external ego-protective buffer may only be

successful in environments that make two things explicit: (1) the learner has many

options to control her own performance and (2) the learner’s performance is expected

to change. These two criteria could encourage partial internal attributions to effort

rather than ability. Effort, as discussed earlier, is a much healthier internal attribution

because it is unstable and controllable, while ability, which is stable and

uncontrollable, should not enhance persistence. The learning environment used in this

dissertation study makes the controllability and instability of outcomes explicit.

In the next chapters of this dissertation, we will give concrete examples of ego-

protective buffers that were provided to students in the context of learning

interventions. The intervention we propose is designed to elicit attributions for failure

that are simultaneously internal and external, which should act as a type of ego-

protective buffer, protecting students’ sense of competence. We hypothesize that

provision of an ego-protective buffer will discourage students from giving up after

failure.

13

Understanding Productive Persistence

Persistence in the face of failure is critical, but sometimes students exert effort

on the wrong things. Often when students are motivated to persist in the face of

failure, they simply “try harder” or “work longer,” when it might be more effective to

try a new tack (Kluger & DeNisi, 1996). Intuitively, we know that good learners

persist productively after learning setbacks. When trying again or trying harder

doesn’t work, they may attempt a new strategy, seek help, or switch to a more

achievable learning goal. Productive persistence occurs when, after failure, students

choose the strategies and behavioral moves that drive towards successful completion

of the learning goal. But knowing which learning path to choose following failure is

not a simple task; it requires more than the desire to persist. It takes a sophisticated set

of self-regulated learning skills.

Self-regulated learning

The literature on self-regulated learning – though not focused on failure, per se

– explores the space of possible learning moves one can take when directing her own

learning. Self-regulated learning (SRL) is the process by which learners regulate their

cognition, behavior, and affect to meet their learning goals (Zimmerman, 1989). Self-

regulated learners plan and set goals, monitor their progress towards them, deploy and

adjust strategies accordingly, and reflect on the outcomes (Pintrich, 2004). For

example, when writing an essay, a good self-regulated learner might begin by defining

the main point of the paper, setting a goal of writing to support this point. The learner

might then select a strategy of writing a meticulous outline, planning the overall flow

of the argument and the content of each paragraph. As the learner progressed in her

writing, she would monitor her progress towards the goal of the paper and change her

process accordingly. For instance, if she noticed that the argument of the paper didn’t

quite hang together, she might redefine the main point of the paper. Finally, a good

self-regulated learner would evaluate and reflect on her final writing product. If the

final essay was not as persuasive as she had hoped, the learner might set herself a new

goal of gathering more evidence for her argument and begin the SRL cycle again.

14

The above example illustrates the four phases that are most common in SRL

models: forethought/planning/activation, monitoring, control, and reaction/reflection

(Pintrich, 2004). During the first phase, self-regulated learners set their goals, activate

prior knowledge and metacognition, and plan how they will spend their time. In the

monitoring phase, they take stock of their prior knowledge and determine whether it

will be appropriate for the task. In the control phase, learners select appropriate

strategies for meeting the goal. Finally, during the reaction/reflection phase, learners

evaluate their progress and perhaps cycle through the SRL phases again. Of course,

these phases are not always attempted in a linear fashion, and effective regulation of

learning may require adaptive selection of the appropriate phase, depending on the

outcome of the learner’s evaluation.

We are interested in self-regulated learning when the outcome of the learner’s

evaluation is negative. Once a student has reflected on the outcome and discovered

that she did not meet her learning goal, what does she do next? SRL models are not

clear on how a “good” self-regulated learner responds to negative feedback. How

does she go about navigating the phases? Does she choose to reset her goal, adopt a

new strategy, or continue to monitor her performance, in an attempt to diagnose the

problem? Does she engage in certain patterns of phase shifts? Unfortunately, few

models focus on self-regulated learning following failure, though some have attempted

to incorporate the processing of feedback (Butler & Winne, 1995). What students

actually do after failure is an open question, and this research aims to answer it.

Another question is how should students regulate their own learning after

failure in order to maximize learning? It is difficult to answer this question because

much of the SRL literature is focused on the elaboration of theoretical models rather

than empirical data collection (Winne, 2001). And of the empirical studies that do

exist, relatively few of them measure actual self-regulated learning behaviors. The

most common measures of SRL are surveys and interviews that rely on self-reports,

which may drastically differ from what people actually do in the moment (Pintrich et

al., 1993; Weinstein, Schulte, & Palmer, 1987; Zimmerman & Martinez-Pons, 1986).

There are a few think-aloud (Azevedo, Guthrie, & Siebert, 2004) and observational

15

measures (Perry, 1998; Turner, 1995) that capture actual SRL behaviors made in the

moment, but they are rare (Winne & Perry, 2000).

Computer-based measures of SRL: Examples from the help-seeking literature

A relatively new type of SRL measure involves tracking student actions in

computer-based learning environments. These new measures may lead to new

discoveries of how students regulate their learning after failure. Computer-based

measures of SRL show promise for several reasons. First, they measure actual SRL

behaviors rather than self-reports of behaviors. Second, they track SRL behaviors as

they occur, rather than retrospectively or prospectively. Third, they add a level of

precision to measures of SRL, since every “click” can be captured, documenting very

precise behavioral moves rather than general study habits. Fourth, they can capture

sophisticated sequences of behaviors that may comprise complex patterns of SRL

(such as navigating between the SRL phases). Most importantly, these measures can

tie behaviors during the learning activity to actual learning outcomes, allowing for

very precise titration of the relationship between SRL and learning. Computer-based

measures of SRL could prove very fruitful for describing the types of learning moves

students make after failure and determining which ones are productive. Moreover,

these findings could inform the design of interactive learning environments, where the

computer system could detect and respond to SRL behaviors.

The best example of successful computer-based measures of SRL comes from

the literature on help-seeking in intelligent tutoring systems. While help-seeking is

not the focus of the current work, the literature can clarify how computer learning

environments can advance our understanding of SRL. Help-seeking can be viewed as

a self-regulatory skill that involves all the components of self-regulated learning such

as monitoring progress to determine when to seek help, setting a goal for help-seeking

outcomes, selecting an appropriate help-seeking strategy, and evaluating the

effectiveness of the help (Nelson-LeGall, 1981; Newman, 1994). While most studies

of help-seeking in interactive learning environments have found that relatively few

students actually engage in help-seeking behaviors, findings have revealed some

16

interesting new phenomena (Aleven & Koedinger, 2000; Aleven et al., 2003; Mandel,

Grasel, & Fischer, 2000).

Most computer-based studies of help-seeking take place in the context of

intelligent tutoring systems. An intelligent tutoring system is a computer-based

learning environment where a learner is coached by a computer tutor as she solves

multi-step problems, usually in math domains. Most tutors offer on-demand help

systems, which allow the learner to ask for a “hint” at any time during the problem-

solving process. Hints get progressively more explicit as the student continues to ask

for help on the same problem. For instance, in an algebra tutor, the first hint might

refer the student to the relevant equation, a second level hint might show the student

an operation on that equation, and the final, “bottom out” hint would plug in the

numbers to the equation and show the correct answer. The term help-seeking, as

discussed here, encompasses all the intricacies of when, how, and why the student

requests a hint while solving a problem in an intelligent tutoring system.

Behavioral measures of this self-regulated learning skill have run the gamut

from very basic to extremely sophisticated. Regardless of where they sit on this

spectrum, measures of help-seeking behavior have proven to be fairly precise

predictors of learning outcomes. To illustrate the variety of help-seeking measures

and their predictive power, we review a selection of studies from the help-seeking

literature.

Wood and Wood (1999) examined the help-seeking behaviors of 14 and 15

year-olds as they worked with an algebra tutor called QUADRATIC. They found that

63% of the variance in posttest scores was predicted by a combination of number of

errors made, number of helps requested, and time spent per operation. When pretest

scores were partialled out, time per operation and time spent before asking for help

predicted post-test scores. An analysis of when the help occurred found that seeking

help after failing a problem step was associated with greater gains from pre to posttest.

Moreover, low gainers used the help less effectively; they often asked for help before

attempting a step, even though help after failure benefited them more. This work

showcases various types of help-seeking measures, such as number of helps, time

17

spent before help, and when help is sought. It also demonstrates how nuanced

measures of SRL behaviors can begin to uncover the complex interplay between SRL

and learning. Finally, it hints at the importance of considering failure in conjunction

with SRL.

Other work has demonstrated the importance of individual differences in help-

seeking. For instance, Renkl (2002) had adults work with a probability tutor called

SEASITE which offered both minimal and extensive explanations. He used cluster

analysis to identify four types of help users. Successful rare users had high prior

knowledge and scored well on the posttest, even though they accessed the help

features infrequently. Unsuccessful rare users had average prior knowledge but scored

poorly on the post-test and so presumably would have benefited from greater use of

the help. Mediocre users started off with average prior knowledge, asked for mostly

minimal explanations, and scored average on the post-test. Finally, successful users

started off with very low prior knowledge but asked for frequent extensive

explanations and reached an average score on the posttest. These analyses underscore

the importance of individual differences, such as prior knowledge, which are critical

factors in defining appropriate help-seeking behavior. Moreover, they add complexity

to the help-seeking picture – the type of help-seeking one engages in (e.g. seeking

minimal versus extensive explanations) influences learning.

Other work has found that progressions in help-seeking behavior indicate use

of broader self-regulated learning skills. For instance, in a study by Wood (2001),

children who used “frequent and deep levels” of hints in the EXPLAIN tutor learned

less than children who used the help as a scaffold, starting with deep level hints and

then fading this help-seeking as they became more practiced in problem-solving.

Likewise, Shute and Gluck (1996) had adults learn about electricity with the Ohm

Tutor, which offers a variety of different help tools. They found that a pattern of high-

then-low tool use was most productive, such that students who started with high tool

use and later progressed to low tool use had the greatest learning gains.

Finally, these types of behavioral measures of learner actions can reveal some

surprising findings. For instance, while many studies have found very low rates of

18

help-seeking behavior, Baker et al. (2004) discovered that a significant portion of

students engaged in what he called “help abuse.” These students “gamed the system”

by rapidly asking for every single problem hint until the tutor gave them the answer.

Baker et al. (2004) originally measured “gaming” behaviors (among other types of

behaviors) by observing students in the classroom for 20-second intervals. They

found that gaming and “talking on-task” behaviors were negatively correlated with

posttest scores, even when pretest scores were accounted for. In later work, Baker’s

group used a machine-learning algorithm to automatically detect gaming behaviors in

real time (Baker, Corbett, & Koedinger, 2004). In this case, computer-based measures

of SRL revealed some unanticipated and very poor SRL behaviors that greatly reduced

learning.

Based on Baker et al.’s work and other related findings (e.g. Aleven McLaren,

Roll, & Koedinger, 2004), Roll, Aleven, McLaren, & Koedinger (2007) built the Help

Tutor, which provides students with help-seeking hints as they solve geometry

problems. Unfortunately, the Help Tutor did not improve learning nor impact future

help-seeking behaviors. This is perhaps because the design of the Help Tutor did not

address student motivation in response to negative feedback about help-seeking errors.

A critique of help-seeking studies in intelligent tutoring systems

One critique of intelligent tutoring systems is that they do not take affective

variables like motivation into account when designing their systems. For instance,

help abuse may be a type of self-defensive behavior where students avoid failure by

simply asking for the answer (in these environments students cannot opt out of

problems they are assigned). The Help Tutor does not attempt to boost students’

motivation after receiving negative feedback nor does it encourage risk-taking. It

simply gives “cold” metacognitive hints without regard for the “hot” aspects of an

environment that may affect learning, including attributions of success and failure.

Therefore, in this study, we will compare SRL behaviors in a computerized learning

environment both with and without an ego-protective buffer. By hypothesis, this will

permit an investigation of how failure attributions influence productive persistence.

19

Another limitation of the computer tutoring studies is that they are confined to

one particular self-regulatory skill: help-seeking. This is partly an artifact of the

intelligent tutoring environment. Like a human tutor, an intelligent tutor provides

guidance by selecting problems for the student and even constraining her problem-

solving steps. A side effect of this guidance is that it greatly restricts the amount of

self-regulated learning the student can engage in. Apart from requesting hints, the

student has very little control over her own learning. However, most self-regulated

learning necessitates a tremendous amount of choice for the learner. According to

Zimmerman (1994, p. 7), “the criterion of personal choice or control is essential to the

exercise of self-regulation.” Environments that provide students with more learning

choices should provide a more complete picture of self-regulated learning that go

beyond single skills like help-seeking. For this study, we have designed a computer-

based learning environment that provides the learner with an extensive array of

choices ranging from the topic to the method of learning to the frequency of

evaluation. This will make it possible to look at sequences of choices rather than the

mere frequency of a particular choice, which in turn, should yield patterns of

productive persistence.

Conclusions

We have argued that failure is an inevitable part of learning that has both a

positive and a negative side. On the positive side, people can learn a great deal from

their mistakes, and some pedagogical styles and classroom interventions have

capitalized on this idea. On the negative side, failure can have negative consequences

for student motivation and may lead students to quit the task entirely, which impedes

learning.

Persistence after negative evaluations is critical for continued learning. One

factor that affects resilience after a learning setback is how people perceive the failure.

For instance, attributing failure to uncontrollable and internal causes like innate ability

provides no incentive for students to persist. Many attribution retraining studies

encourage students to attribute failure to low effort. We have argued for a different

20

type of attribution-based intervention in the form of an ego-protective buffer. An ego-

protective buffer helps an individual maintain a stable sense of competence in the face

of failure. The type of ego-protective buffer tested in this study invites students to

attribute failure to a combination of internal and external sources. We hypothesize

that external attributions will shield the learner’s ego from blame while internal

attributions will place the onus of remedying the failure on the student herself,

particularly in an environment that provides explicit ways to exert effort towards the

task (i.e. many learning resources and choices) and expectations that performance will

change. This study tests the impact of an ego-protective buffer on persistence after

failure.

Despite being persistent, learners may still select unproductive learning paths

or perseverate on ineffective strategies. The literature on self-regulated learning,

while not focused on failure per se, provides some guidance in describing the types of

learning behaviors students might take after negative evaluations. For instance, the

SRL literature offers many detailed models of how learners set goals, monitor their

progress, and select strategies for learning. Unfortunately, there is little empirical

evidence linking specific self-regulated learning behaviors with learning outcomes, in

part because of the lack of behavioral measures of SRL. Computer-based learning

environments offer precise, in-the-moment measures of SRL behaviors and have been

used extensively to measure help-seeking – a specific self-regulatory skill. Findings

from this research reveal some interesting relationships between SRL and learning,

though they do not relate directly to failure and the scope of the findings is limited by

the lack of student choice in many learning environments.

The study we describe in Chapter 3 explores students’ self-regulated learning

behaviors in a computer-based learning environment riddled with learning choices.

Analyses will focus on identifying which post-failure SRL behaviors are productive,

by relating them to learning outcomes. Furthermore, we will examine whether

provision of an ego-protective buffer can reduce one particularly unproductive

behavior – quitting after failure.

21

CHAPTER 2: RELEVANT PRIOR RESEARCH

In this chapter, we describe three experiments that provide the rationale for the

hypotheses in the proposed study. In the first two experiments, we contrasted two

conditions. In the Teachable Agent (TA) condition, children were given the goal of

teaching a digital pupil (a Teachable Agent) in the context of the Betty’s Brain

software learning environment. In the Self condition, students were given the goal of

learning with the aid of the same software. In Study 1, TA students chose to spend

more of their time on learning activities and they also learned more. In Study 2, TA

students were more likely to express negative affect and made a mix of internal and

external attributions for failure. They also persisted longer at the learning task,

particularly after receiving negative feedback. We hypothesize that TA students were

more persistent in the face of failure because the agent (the digital pupil) provided an

ego-protective buffer, which allowed students to blame some of the failure on the

agent, while accepting some of the blame themselves. In this dissertation study, we

explicitly test the effect of an ego-protective buffer on persistence after failure.

In study 3, eighth grade students engaged in self-regulated learning in the

context of a choice-filled genetics game. The game was carefully designed to provide

learners with many learning choices, which would allow them to engage in

unrestricted self-regulated learning. In this exploratory study, we asked which self-

regulated learning behaviors were most closely associated with learning outcomes.

An interesting finding was that the number of times students failed a level and then

abandoned it entirely (without checking a relevant help resource or playing the same

level again) was negatively predictive of learning outcomes. We hypothesize that

students who fail-abandon are attributing failure to internal, uncontrollable causes

(like low aptitude), and this leads them to give up the task entirely. Provision of an

ego-protective buffer – if effective at increasing persistence – should decrease this

fail-abandon behavior and increase learning gains. The current study will explore this

hypothesis and attempt to replicate the findings of Study 3.

22

Studies 1 and 2: Teachable Agents and the Ego-Protective Buffer

A Teachable Agent called Betty’s Brain

In both studies 1 and 2, students were learning in the context of Betty’s Brain.

Betty’s Brain is a type of Teachable Agent software where students learn by teaching

a graphical character on the computer (Biswas et al., 2005; Schwartz et al., 2007).

Each student creates her own “agent” or digital pupil and then populates its “brain”

with knowledge. Students teach their agents by building concept maps of nodes

connected by qualitative causal links; for example, ‘heat production’ increases ‘body

temperature’ (see Figure 1). Betty’s Brain is designed to help students learn causal

thinking in science domains.

A Teachable Agent (TA) is equipped with an artificial intelligence reasoning

engine which enables it to reason logically through the links it has been taught. For

instance, a TA can answer questions about the relationship between two concepts, as

demonstrated in Figure 2.1. In response to a query, the TA will respond by

successively highlighting each node and link in a causal chain as it reasons through

them. This makes the TA’s “thinking” visible to the student, who can revise her

agent’s knowledge by editing its concept map; meanwhile, the student herself learns

along the way.

23

Figure 2.1. Screenshot of Betty’s Brain concept-mapping interface. In the bottom left corner is Rockstar, a student’s agent. A sample student concept map of fever mechanisms is in the center of the frame. In the pop-up to the right of the screen, the student has asked her agent a question about the relationship between ‘temperature set point’ and ‘heat production.’ The agent has responded by highlighting its reasoning in the concept map and stating its answer in the Talk Log box.

Betty provides students with several optional learning activities that are

designed to help students understand the science content. A reading resource provides

background information on the causal relationships in the map. The map-building

features enables students to construct a diagram of their knowledge. In addition to the

query feature discussed above, there is also a quiz feature which allows students to

receive feedback on their maps. These various features are meant to engage students

in a cycle of reading, map-building, feedback, and revision.

Betty also comes with some play features like the Triple-A Game Show

displayed in Figure 2.2 – a Jeopardy-like game where students’ agents play against

one another. During the game show, the host poses a series of questions. Students

wager points on their agent’s answer while the host provides right/wrong feedback and

awards points. Betty also contains a chat feature where students can carry on a written

conversation with one another while working in the Betty software.

24

Figure 2.2. Screenshot of the Triple-A Game Show and chat window. Agents play against other agents in a Jeopardy-like game. The host poses questions which the agents answer based on their maps. The students participate by wagering points on their agent’s answer. The bottom of the screen displays the chat feature, which students can access anytime they are in Betty’s Brain.

Past studies found that Betty’s Brain led to greater learning gains than standard

practice, standard concept-mapping tools, and intelligent tutor versions of Betty

(Biswas et al, 2005; Chin et al., 2010). Anecdotal evidence from these studies

indicated that students were highly motivated to use the Betty’s Brain software and

perhaps cultivated a sense of responsibility for their agents’ learning. We wondered

whether this sense of responsibility towards another motivated students’ learning.

Studies 1 and 2 sought to address the motivational impact of social interaction with a

tutee during the process of learning. Are students more motivated to learn when they

learn for the sake of another than for themselves, and will this affect learning? Will

students respond differently to feedback when it goes toward an agent?

25

Study 1: Do students learn more for the sake of another?

In this study, 8th grade students used the Betty’s Brain software to learn how

the body generates a fever (Chase, Chin, Oppezzo, & Schwartz, 2009). The main

difference between conditions was merely the goal we gave to students on the first day

of instruction: teaching versus learning. One group of students (the TA group)

believed they were teaching their digital pupils while a second group (the Self group)

believed they were learning for themselves. TA students were told: “today you are

going to teach your agent about how the body generates a fever.” They were

instructed to “teach” their agents by creating a concept map that would represent the

“agent’s brain.” In contrast, the Self group was told: “today you are going to learn

about how the body generates a fever” by building concept maps, which was a fairly

typical learning activity for these students. Both groups used nearly identical versions

of the Betty software with an on-screen character who represented either the tutee or

the student herself, depending on condition. Sixty-two students from four different

classes participated in the experiment; intact classes were randomly assigned to

condition.

Students used Betty in the classroom over two days of instruction. For the

most part, students were given free reign to regulate their own learning; they chose

how to spend their time in the system. They could read the resources, edit the map,

take quizzes, query the map, play the game show, or use the chat tool whenever they

wanted. Students tended to bounce back and forth between these activities. Every

student action within the system was logged on a server, leaving a record of student

behaviors and time spent using various Betty features.

Students in the TA group chose to spend more of their time on learning

activities. About 50% of their time in the system was spent taking quizzes, asking

questions, editing the map, and reading. In contrast, the Self group spent most of their

time playing the game show and chatting; only 20% of their time was spent on

learning activities. Looking at reading times alone, the TA group spent significantly

more time reading (MTA = 13.4 mins, MSelf = 8.4 mins), which is particularly

impressive given the presence of other, more attractive options (e.g. games, quizzes)

26

in their environment. These results demonstrate that TA students persisted longer at

Betty’s learning activities which suggests that they were also more motivated to learn.

Moreover, on a paper-and-pencil post-test of factual, integration, and

application problems, the TA group significantly outperformed the Self group on the

“harder” questions – the integration and application problems. In Figure 2.3, the

groups are split into high and low achievers based on prior science grades,

demonstrating that the TA was particularly effective for low-achieving students. In

fact, on harder questions, the low achieving TA students performed at the same level

as the high achieving Self students.

Figure 2.3. Posttest scores broken out by item type, achievement level, and condition. The low-achieving TA students performed as well as the high-achieving Self students on inference and application questions. Figure adapted from Chase, Chin, Oppezzo, & Schwartz, 2009. In this study, learning on behalf of a TA inspired persistence at learning-

relevant activities and resulted in greater learning gains. The next study explored

possible motivational mechanisms behind this effect. We believed that part of these

effects were a function of watching feedback go to another versus oneself. We

wondered whether the agent could act as some sort of buffer for the negative

ramifications of failure. This would explain why low achievers, who experience

frequent failure and stand to gain more from an ego-protective buffer, showed greater

learning gains. So in this next study, we paid special attention to students’ reaction to

failure feedback while playing the game show.

27

Study 2: Does learning for the sake of another impact motivation in the face of failure?

Twenty-four 5th grade students were pulled from class for individual think-

aloud sessions, while they worked with Betty’s Brain (Chase, Chin, Oppezzo, &

Schwartz, 2009). This study contrasted the same two conditions and taught the same

science content. But this time, students provided think-alouds while they worked with

the software, which were analyzed for evidence of affect and attributions for failure.

Persistence times were collected and it was noted whether students chose to revise

their agent’s (or their own) knowledge following feedback.

The study took place in an hour-long session comprised of three phases: Study,

Play, and Revise. During the Study phase, students read a passage about fever

mechanisms then built concept maps to organize their knowledge on the topic. The

TA group was told that the purpose of building the map was to teach their agent, while

for the Self group, the object of the concept mapping activity was to learn (for

themselves).

After students built their maps they moved on to the Play phase where they

played one round of the game show alone. In the TA group, the agents answered the

host’s questions based on their maps, while in the Self group, the students themselves

answered the questions by selecting answers from a drop-down menu. Game show

questions ranged in difficulty, ensuring that each student would experience a

combination of success and failure.

Think-aloud protocols during game show play revealed that TA students were

much more likely to acknowledge failure than Self students. After getting a question

wrong in the game show, TA students made far more spontaneous statements of

negative affect (“I’m sorry Diokiki” or “Ungh!! Why does he keep answering large

increase!?!”). They also made more attributions of blame for the failure (“I didn’t

know this one” or “He got it wrong”) whereas the Self students rarely mentioned

failure. The TA students showed greater attention to failures by expressing negative

affect and making attributions.

An examination of how the TA students spontaneously apportioned blame for

failure revealed an even spread of attributions across the student and the TA. About

28

28% of attributions were made towards the TA (“He got it wrong”), 32% were

directed to the self (“I didn’t understand that question”), and 40% were ascribed to

some combination of both (“I, err… he didn’t know this one”). In contrast, Self

students, who did not have the luxury of an agent scapegoat to take the fall for them,

made 100% of attributions to themselves. For TA students, the agent created an outlet

for failure blame, yet the students themselves took some responsibility by making just

as many self-attributions.

Furthermore, TA students acted on this sense of responsibility by choosing to

revise their understanding. After receiving feedback in the game show, students had

the option of revising their understanding in preparation for a second, harder round of

the game. During revision, students were allowed to review the passage, view and edit

the concept map, or look over the game show feedback. A full 100% of TA students

chose to revise after the game show compared to only 64% of Self students.

In some sense, TA students’ drive for revision is not surprising, given that the

TA’s performance is dependent on the maps. But this aspect of teaching a TA is an

important part of moving beyond failure. For persistence to seem fruitful, there must

be clear actionable paths or possible approaches for improvement. For the TA

students, it is obvious how to increase the TA’s knowledge – fix the links in its brain.

For the Self students, it may not be so obvious how they can increase their own

knowledge, especially for young children who do not have well-developed SRL skills.

It is notable that 36% of Self students did not choose to revise at all; they did not even

see the value in glancing at the reading again. And of the Self students who did

choose to revise their knowledge, they spent a mere 1.5 minutes doing so, compared to

the TA students’ average of 8.6 minutes. Perhaps by having the means for repairing

the failure at their fingertips, TA students were willing to put in the effort to persist.

We posit that presenting students with the narrative of teaching an agent

provided them with an ego-protective buffer that motivated persistence after failure.

The agent occupies a unique social space of part self, part other. As a product of the

child’s tutelage, the agent is a reflection of the child’s knowledge. But by taking

independent actions (e.g. answering questions, taking quizzes, playing the game

29

show), the agent takes on a life of its own. In failure situations, this in-between-self-

and-other quality of the agent makes causal attributions ambiguous.

Since the TA was the one performing in the game show, not the students

themselves, the TA could absorb part of the blame for failure, sparing students’ sense

of competence. At the same time, students felt responsible for their agents’

performance, since it was enacting their teachings. This combination of internal and

external attributions may have spurred the students to continue work on the task.

Moreover, students have explicit control over their TA’s performance by

essentially programming its knowledge. Also, Betty’s Brain stresses the unstable and

changeable performance of the agent. After all, students literally edit their agent’s

“brain,” watching it change and grow. So in addition to the ego-protective buffer, the

conditions of controllability and instability – two dimensions of causal attributions

that provoke persistence – are incorporated into the teaching narrative of Betty’s

Brain.

Perhaps a confluence of these three elements – an ego-protective buffer, clear

control over outcomes, and an expectancy that performance will change – ultimately

led the TA students to acknowledge failures and attend to those failures by persisting

at the task. Since all three of these elements were simultaneously provided by the

agent, this study does not isolate the effect of the ego-protective buffer on persistence.

The dissertation study (described in Chapter 3) manipulates the presence of the ego-

protective buffer only, while holding other variables constant.

Study 3: Fail-abandon – A Poor Self-Regulated Learning Behavior

Study 3 was designed to achieve two broad goals that relate to the proposed

study: (1) to measure and explore self-regulated learning behavior and (2) to

empirically derive the types of self-regulated behaviors that lead to learning. To allow

for complete self-regulation of learning, we designed a new learning environment that

provided students with a tremendous number of learning choices. Students could

choose both the topic they wanted to work on and the resources that would help them

learn. They could also choose when to continue learning about a topic and when to

30

move on to another topic. Students were pulled from class in groups of 6 to play the

game independently for one 35-minute session.

The environment we designed was an educational game about genetics. The

game contained 5 different cumulative levels. Each level was designed to address a

different, but related genetics topic, including Punnett squares, probability, family

trees, co-dominance, and the inheritance of multiple genes. Levels were cumulative

and became increasingly more difficult, such that Level 5 was the hardest and tested

information from levels 1-4. Each level of the game contained several rounds of equal

difficulty, which allowed students to play the same level several times.

Each round of a game contained several genetics questions. When students

completed a round, the experimenter checked their answers and returned the marked

round back to students so they could see which answers were right and wrong.

Students were awarded points for rounds that earned a score of 75% or more questions

correct. When students got fewer than 75% of the questions correct in a round, they

lost a “life”. Points and lives were included to give the task a more game-like feel and

to motivate students to make choices carefully. For the purpose of our analysis,

winning points was treated as a “success” while losing a life was treated as a “failure.”

Throughout the game, students were free to make many choices. For instance,

they could select which level they wanted to play or whether they wanted to inspect a

help resource. Help resources included level-specific readings, solutions from rounds

played, and a glossary of vocabulary terms. Another choice included a distractor task

in which students could draw their own alien.

Each “choice” was represented by a different sheet of paper. Each student had

her own set of folders containing rounds for each level of the game, readings for each

game level, solutions to game rounds played, a glossary of terms, and the alien-

drawing task (see Figure 2.4). Every time a student wanted to make a different

“choice,” she would pick a new sheet of paper, and when she was finished with that

choice, she would place the sheet of paper face down in a “discard” pile. At the end of

the session, we collected these piles of paper, which represented our “clickstream”

data of the sequence of choices that were made.

31

Figure 2.4. Set-up of learning choices for the Scaffolded condition. Each folder contained sheets of readings, solutions, and game rounds specific to each level of the game. “Help” resources are outlined in bold. In the Unscaffolded condition, the “levels” were not labeled.

A side question, which proved to be less interesting in the end, was whether

provision of scaffolding would affect students’ choices within the environment and

resulting learning outcomes. Students in the Scaffolded group were told about the

cumulative leveling of the game. For them, the various levels of the game were

labeled with the level number (e.g. Level 1, Level 2, etc.). The Unscaffolded

condition was not told about the game’s built-in levels. Their folders were labeled

only with the game topic (e.g. Punnett squares, etc.). The game instructions sheet

explained that some topics were more difficult than others, but it gave no indication of

which ones were harder or easier. A Control condition did not play the game but

received regular instruction (on an unrelated topic) from their classroom teacher. All

three conditions took pre and posttests on the genetics content taught in the game. We

32

hypothesized that the Scaffolded group would learn the most from playing the game

because they would play the prescribed levels in order, and a more sequenced

presentation of the information should help students build a better conceptual

understanding of the material.

Both Scaffolded and Unscaffolded groups learned from playing the game, as

evidenced by their superior test performance in comparison to the non-treatment

control group. However, there were no learning differences between Scaffolded and

Unscaffolded conditions. We also did not find any drastic differences in students’

self-regulated learning behaviors across the conditions. The only major difference

between groups was in their leveling behavior. As predicted, the Scaffolded group

tended to “level up” by starting at level 1 and moving their way up to level 5

sequentially, one level at a time. In contrast, the Unscaffolded group, which was

unaware of the leveling scheme, jumped randomly from level to level, not following

any particular pattern. This difference in choice patterns was expected, given the

design of the learning environment, however, we did not find significant differences in

any other choices, such as the number of games played, the number of help resources

accessed, or the number of game failures.

However, we did find differences between conditions in how their use of help

features related to learning outcomes. In the Scaffolded condition, use of the help

features was unrelated to performance on the posttest. But in the Unscaffolded

condition, several measures of help usage were correlated with learning outcomes (see

Figure 5). For instance, the number of “relevant helps” (accessing either a reading or

a solution on the same level as a recent game) was moderately correlated with posttest

scores (r = .52), even when pretest and number of in-game failures were partialled

out. When the relevant helps category was further subdivided into time-ordered

transitions of “help-game” and “game-help” counts, only the game-help frequency was

correlated with learning (r = .42). Finally, we broke the game-help category into helps

accessed after failure (fail-help) and after success (success-help). Only the number of

fail-helps was significantly correlated with posttest scores (r = .47). This suggests

that help-seeking after failure is particularly important for learning.

33

Table 2.1. Partial correlations with posttest for the Unscaffolded condition, removing variance due to pretest and number of fails. No. of Relevant Helps .52** No. of Relevant Help-Game transitions .22 No. of Relevant Game-Help transitions .42* No. of Relevant Success-Help transitions .18 No. of Relevant Fail-Help transitions .47* No. of Success-abandon transitions .02 No. of Fail-abandon transitions -.45*

* p < .05, ** p < .01

Moreover, when we examined what Unscaffolded students did after failure

when they were not accessing a relevant help resource, we found that 91% of the time,

they abandoned the failed level and jumped to a new level entirely. This fail-abandon

behavior was negatively correlated with post-test scores (r = -.45). Furthermore,

when the variables listed in the Table 2.1 (along with pre-test scores and total number

of failures) were placed in a stepwise regression, the only variable that entered the

equation was the number of fail-abandons (ß = -.14, p < .001). This variable alone

explains 56% of the variance in post-test scores. Interestingly, of the 21 students in

the Unscaffolded condition who experienced failure, about half of them (10 students)

engaged in at least one fail-abandon behavior, and these 10 fail-abandon students

learned considerably less than the 11 fail-persist students (t (19) = 2.50, p = .03).

We became interested in the utter lack of persistence that some learners in the

Unscaffolded condition displayed in response to failure while others engaged in

productive persistence by adopting healthy SRL behaviors like help-seeking. One

hypothesis is that learners who abandoned a game after failure were engaging in self-

protective behavior. Perhaps they attributed failure to low aptitude and therefore, had

no hope of ever succeeding in the game, and so avoided future failure by abandoning

the level altogether. This type of ego-protective response might be blocked by

provision of an ego-protective buffer. The next study asks whether provision of an

ego-protective buffer embedded in the game could reduce this fail-abandon behavior,

encourage healthier SRL behaviors, and improve learning outcomes. The next study

also engaged students in longer periods of game play in hopes of uncovering longer

sequences of behaviors which could map onto larger-scale SRL patterns.

34

CHAPTER 3: METHODS

Study Overview

The current study was designed to follow up on two findings from prior

research. The first finding was that giving students the goal of teaching an agent

increased their learning, persistence, and attention to failure, relative to students who

were learning for themselves. One explanation for this effect is that the agent

provided an ego-protective buffer, which enabled students to persist in the face of

failure. However, in these prior studies, provision of an ego-protective buffer was

confounded with many other differences across conditions. The current study is meant

to isolate the effect of the ego-protective buffer on persistence and learning.

Moreover, it tests the generalizeability of this effect by instantiating the ego-protective

in a different context and a novel implementation. We hypothesized that the ego-

protective buffer would increase persistence after failure by reducing the fail-abandon

behavior exhibited in prior work.

The second finding, from the prior SRL study, was that persistence after failure

was a strong predictor of learning outcomes. The prior study was implemented with

paper-based choices and close oversight by the experimenter. In the dissertation

study, the same genetics game was implemented on the computer and played in a more

ecologically valid classroom setting. These two differences could have a large impact

on students’ learning choices, but still, we expected the fail-abandon behavior to be

robust. We also sought to gather a larger corpus of students’ self-regulated learning

choices, to allow fuller exploration of productive persistence.

To summarize, the dissertation study was designed to answer the following

research questions:

1. Does an ego-protective buffer promote persistence after failure?

2. Does persistence after failure influence learning? (Is the fail-abandon behavior

negatively correlated with learning?)

35

3. What constitutes productive persistence after failure? (What kinds of self-

regulated learning behaviors facilitate learning from failure?)

4. Does an ego-protective buffer promote productive persistence?

To test these questions, the current study engaged students in self-regulated learning as

they played a choice-filled, educational computer game. Students played a modified

version of the Unscaffolded game from the prior SRL study. As before, students had

access to multiple rounds of games on different topics in addition to multiple learning

resources. There were two conditions. In the Ego-Protective Buffer (EPB) condition,

students were told success and failure in the game was a probabilistic outcome. The

more questions they got right in a game round, the more likely they were to win, but

there was still an element of chance involved. We hoped this would invite students to

make a combination of internal and external attributions for failure. In the Control

condition, students were told that success and failure in the game was a deterministic

outcome. To win a game round they needed to get 75% percent of answers correct.

We hoped this would incite largely internal attributions for failure. In reality, wins in

both conditions required students to get 75% correct, we only manipulated the EPB

students’ perceived causes of success and failure.

Students in both conditions played the game for two class periods. Internal

and external attributions were measured by interviewing a subset of the students in the

study while they worked with the genetics game. Student “click” data provided a

measure of SRL behaviors including fail-abandon. Pre- and posttests assessed

learning gains due to the treatment. We also collected surveys that measure self-

regulated learning skills and motivational constructs to determine the relationship

between these self-reported individual differences, behaviors in the game, and

learning.

Design of Learning Environment

To address the research questions, we designed a learning environment that

enables students to regulate their own learning, experience frequent success and

failure, and learn complex genetics content. The final product was a genetics game

36

called Mendel’s Galaxy, which was based on the paper prototype game used in the

prior SRL study. To encourage self-regulated learning and to give students a sense of

control over learning outcomes, we provided students with many learning choices. In

addition to choosing learning resources, students selected both the topic(s) they

wanted to learn and the sequence of these topics. To investigate persistence in the

face of failure, we created topics of varying difficulty, to ensure a mixture of success

and failure in the game. We also produced multiple game rounds per topic, to give

students the option of trying again after losing. This was also meant to stress the

malleable nature of student performance in the game, and to create an expectation of

continued improvement. Also, because we were interested in the fail-abandon

behavior that appeared in the pilot study’s Unscaffolded condition, we chose not to

label the implicit “levels” of the game, replicating the Unscaffolded condition.

Instead, we allowed students to explore the topics on their own, hoping they would be

able to discover the levels for themselves.

Mendel’s Galaxy is built around an alien genetics narrative. Players take on

the role of geneticists-for-hire who travel the galaxy, hopping from planet to planet,

helping aliens unlock the secrets of their heritage by solving a series of genetics

puzzles (i.e. leveled games). Figure 3.1 shows a screenshot of the game homepage

and its seven planets, each of which covers a different genetics topic.

37

Figure 3.1. Homepage of Mendel’s Galaxy – an alien genetics game for middle schoolers. Each planet addresses a different topic and contains resources to help students learn the topic. There are a total of 4 isomorphic puzzles for each topic, which are similar to rounds of a game. The on-screen placement of planets was randomized for each player, to ensure that any pattern in the sequence of “levels” attempted was not driven by a planet’s location.

Mendel’s Galaxy was designed to teach students about the mechanics of

dominant and co-dominant inheritance. The game covers seven different topics

including: Punnett squares, probability, prediction, family trees, co-dominance,

Punnett squares with two genes, and family trees with 2 genes. Students can

investigate each topic by traveling to a planet. We developed a different puzzle for

each topic to give students practice in applying various genetics concepts. The topics

are leveled, such that information from previous topics is necessary to solve the

puzzles of a later topic, making some topics’ puzzles much harder than others. To

mimic the Unscaffolded condition from the pilot study, students were not told about

these implicit “levels.”

38

Each planet (topic) contains 4 isomorphic puzzles or game rounds, to provide

students with multiple chances to master a topic. Each puzzle contains a set of

questions or “challenges.” The number of questions on a puzzle ranges from 4 to 10,

depending on the topic. Figure 3.2 contains an example puzzle on the topic of family

trees. In this puzzle, students must enter the phenotypes and genotypes for the missing

individuals in the family tree. More example puzzles are available in the Appendix.

The game contains a hierarchy of choices, with some choices nested within

others (see Figure 3.3). On the first choice tier, students choose which of the 7 topics

(or planets) they want to work on. Next, students go to the second choice tier, where

they can select from a set of four resources to help them learn about that topic. They

can “preview” a practice puzzle for a given topic to determine whether they want to

Figure 3.2. Example puzzle from Planet Scila on the topic of family trees.

39

play. They can “read” a textbook-style passage about the topic. They can “play” or

attempt a puzzle (though once all 4 puzzles on a planet are played, students cannot

play any more puzzles on that topic). Or, they can see the correct “answers” to

puzzles already played.

Figure 3.3. Hierarchy of game choices. On Tier 1, students choose a topic. On Tier 2, students choose a resource. Some resources “unlock” access to additional resources on a Tier 3. After each selection, students can return to the homepage to begin the cycle again.

A third tier of choices is unlocked when students select the “play” or

“answers” resources. After students play a puzzle, they are presented with

success/failure feedback and are given the option of viewing the “feedback” resource

that details which problems they got right and wrong. If students choose to view the

correct “answers” to the puzzle, they receive the additional choice of reading

“explanations” for those correct answers. After selecting a resource, students are

typically sent back to the homepage to begin the choice cycle again. Examples of each

of these learning resources are shown in the Appendix. We chose this particular set of

40

resources, because we believed they would support learning of the complex genetics

content and provide a comprehensive data set on self-regulated learning in open-ended

learning environments.

The game allows for frequent experiences of success and failure. For each

puzzle the players win, they are awarded 10 points or “zorks”. Each time players lose

a puzzle, 1 of their “crew” is taken away. The total number of crew and points are

tracked in the score box, located in the upper right hand corner of every screen (see

Figure 1). Once all the crew are gone, the game is over. When the game ends,

students can restart the game. Their point total will return to zero and their crew will

be replenished, but they cannot attempt puzzles they tried in previous rounds of the

game.

The game presented in the current study differed in a few ways from the

prototype game in the SRL study. First, the number of topics was increased from 5 to

7 because we noticed that students had difficulty making certain conceptual “jumps”

between some topics, so we added extra topics to act as conceptual bridges. Second,

we added some additional resources to assess more complex patterns of self-regulated

learning. For instance, we wanted to include some “deep” (answer explanations) and

“shallow” (correct answers only) strategies. Moreover, we wanted to give students the

option to diagnose their errors by viewing the right/wrong feedback. Finally, we

removed a couple of choices that were rarely used in the prior SRL study, such as the

design-an-alien task and the glossary.

Participants

Study participants were 153 seventh-grade students from an extremely

ethnically diverse public school in California (35% Asian, 25% Latino, 22% Filipino,

11% White, 4% African-American and 3% other; 37% qualify for free lunch

programs). Of the 153 study participants, 143 of them completed both days of the

intervention, and of those we received permissions and pre- and posttest data for 130.

Participants were drawn from five different classes taught by the same science teacher.

Participants were assigned to condition through random stratified sampling within

41

class, based on science class grades and gender. It is important to note that students

had some prior knowledge in the domain; they had recently learned the basic

mechanics of dominant inheritance.

Study Design

Manipulation

The study contrasted two conditions: an ego-protective buffer (EPB) condition

and a Control condition. The main difference between conditions was the cover story

explaining how puzzles were won and lost. Even though the computer calculated wins

and losses in exactly the same way, we manipulated students’ perception of how

success and failure occurred.

In the EPB condition, students were told that the outcome of a puzzle was

probabilistic; winning a game was a function of both chance and the player’s

performance. The more questions students answered correctly in a game, the better

their chances of winning. Figure 3.4 shows the page displayed to students after

completing a puzzle. It shows the number of correct answers and the total number of

questions (challenges). To determine the outcome of the puzzle, an alien host throws

green and red balls into a bag. The number of green balls is equal to the number of

correct answers, and the number of red balls is equal to the number of incorrect

answers. The alien host then shakes the bag and randomly picks one ball. If he selects

a red ball, the player loses. If he selects a green ball, the player wins the puzzle. We

hoped this scheme would encourage students to blame failure on a combination of

external (bad luck) and internal sources (their own performance). Moreover, it gives

students an incentive to learn. If they were to get more questions right on the quiz,

their chances of winning would improve.

42

EPB condition Control condition

Figure 3.4. Failure message screens for EPB and Control conditions. The EPB condition believed that the outcome of the game was probabilistic. The Control condition believed the outcome was deterministic.

In the Control condition, students were told that the outcome of a puzzle was

deterministic, based solely on their personal performance in the game. Winning is

determined by a simple 75% rule. If more than 75% of the questions in a puzzle are

answered correctly, the players win. If 75% or less of the questions are answered

correctly, they lose. Since winning and losing in this scenario are entirely dependent

on the player’s performance, we expected students in this condition to make

predominantly internal attributions.

In reality, the computer followed the 75% rule to determine wins for both

groups. However, students were led to believe the cover story in their respective

conditions. Cover stories were explained to students via Powerpoint presentation on

the first day of game play, were reiterated on the second day of the game play, and

were further reinforced by the alien host’s actions in the game.

Another difference between conditions was the circumstances of the alien

host’s dance. In the EPB condition, the alien does a victory dance after a win. In the

Control condition, the alien host throws an equal number of red and green balls into a

bag and randomly chooses one. If he picks a green ball, he will do the dance. This

extraneous detail was included so that both conditions would have an element of

randomness, which often adds to the “fun factor” of games.

43

Condition Assignment

The study was implemented in 5 different science classes. Classes were split

in half based on the random stratified sampling procedure. Half of each class played

the game Tuesday-Wednesday, while the other half played Thursday-Friday. Each

half-class was assigned to a condition as shown in Table 3.1. Conditions alternated by

period to ensure an equal amount of variation in time of day and day of week. Half-

class groups ranged in size from 12 to 17 students.

Table 3.1. Condition assignment by period and day.

Period Tues-Wed Thurs-Fri 1 EPB Control 2 Control EPB 3 EPB Control 4 Control EPB 5 EPB Control

Procedure

Classroom Study

The study took place over the course of one full week of school, in students’

regular classroom. On Monday, all students completed the pretest and presurveys.

Next, students played the game for two 35-minute sessions, spread out over two

consecutive days. Some students played the game Tues-Wed while others played the

game Thurs-Fri. On the days when students were not playing the game, they received

regular instruction on unrelated content, from their classroom teacher. On the

following Monday, all students took the posttest and postsurvey.

On the first day of game play, the experimenter explained the game narrative

and game instructions to the entire treatment group with a Powerpoint presentation.

Each condition received a different explanation of how to win a puzzle. Afterwards,

students logged into the game on individual laptops and played for approximately 35

minutes. To ensure that their self-regulated learning choices were made

independently, students were discouraged from talking to one another.

44

On the second day of game play, the experimenter reiterated the game

instructions and further explained how to win a puzzle. Again, these explanations

differed depending on condition. In the EPB condition, it was explained to students,

through the use of a few examples, that the likelihood of picking a green ball would

increase as the proportion of correct answers rose. In the Control condition, the 75%

rule was reiterated and then demonstrated on the same set of examples.

After the brief explanation, students logged into the game and again played for

about 30 minutes. This time, students started the game afresh. Their scores were reset

to zero, crew were replenished, and all puzzles (including ones the students had

completed the day before) were available for play. This provided a measure of the

topics students would choose on the second day of game play. Would students

attempt puzzles they had failed the day before or would they merely try to repeat their

successes?

Pull-out Study

Two students were pulled from each classroom session to participate in video-

taped interviews as they played the game. These students were pulled after instructions

were given to the whole class, so they heard the same instructions as the students in

the larger classroom study. A total of 20 students, 10 per condition, were interviewed

as they played the game.

Experimenters trained in the interview protocol conducted the one-on-one

sessions. While students played the game, the experimenters occasionally interrupted

with questions that focused on attributions and learning choices. Questions were

asked whenever a student made a new choice, and whenever she experienced success

or failure. Table 3.2 describes the interview protocol.

Table 3.2 Interview Questions Condition Question

Success: puzzle won “Why was that a win?” Failure: puzzle lost “Why was that not a win?” Topic choice “Why did you choose this planet?” Resource choice “Why did you come to this page?”

45

Measures (see Appendix for complete measures)

Pretest

The pretest contained seven items. Each item was designed to address the

content from different planet. The items were very similar to the game puzzles but

used different surface features (i.e. different genetic traits and species) and were

presented in word problem form (rather than the graphic presentation used in the

game). Each item contained several questions using a fill-in-the-blank format. Most

questions asked students to derive the correct genotype or phenotype of a potential

child, given the genotype/phenotype of its parents. Other questions asked students to

calculate the probability that two parents would have a child of a particular genotype.

The first row of figure 7 contains a sample test item.

Each of the seven items contained several sub-questions. Each question was

given a score of 1 for a correct answer and a score of 0 for an incorrect answer.

Because each item contained varying numbers of sub-questions (ranging from 2-8),

scores for each item were converted to proportions (on a scale of 0-1) so that each

item would be weighted equally when aggregated with other items on the test.

Posttest

The posttest contained 7 questions that were identical to the pretest questions

except they differed on the surface features of species and genetic traits. Pre- and

posttests were coded by three separate coders who compared students’ answers to an

answer key. Given the low-inference coding scheme (i.e. answers were either right or

wrong), reliability between coders was not necessary.

Transfer Test

The transfer test contained 4 items, targeting two big ideas that ran throughout

the game: models of inheritance and probability. The 4 transfer items were further

subdivided into near and far transfer items. The 2 near transfer items were word

problems that asked students to apply what they had learned in new ways, but still

within the domain of genetics. The 2 far transfer items also asked students to apply

46

their understandings in new ways, but outside the domain of genetics. Items required

a combination of fill-in-the-blank answers (similar to those on the pretest) and short

answers. See example item types in Figure 3.5.

Once again, items were scored on various scales, ranging from 0-2 to 0-8,

depending on the number of genotypes or probabilities that needed to be calculated for

each item. For the sake of analysis, scores for each question were converted to a 0-1

scale. 25% of the items were scored independently by two coders, and reliability was

very high (92%). Given the high reliability, the remaining 75% of the data were coded

by a single coder.

47

Item Type

Sample Item

Posttest (nearly

identical to

pretest)

Near Transfer

Far Transfer

Figure 3.5. Sample post-test items

48

Pre-survey

The pre-survey assessed several different constructs. A portion of the survey

assessed two constructs of self-regulated learning strategies: cognitive strategy use and

self-regulation. There were 8 items for each construct selected from the Motivated

Strategies for Learning Questionnaire (Pintrich and De Groot, 1990; Pintrich, Smith,

Garcia, & Mckeachie, 1993).

The survey also assessed several other motivational constructs including

mastery, performance-approach, and performance-avoid goals, self-efficacy, interest,

and fixed intelligence. There were 4 survey items per construct, with the exception of

the fixed intelligence scale, which contained 3 items. Items were taken from the

Patterns of Adaptive Learning Scales (Midgley et al., 2000), Dweck (2000), and Lau

& Roeser (2002). All items were adapted to apply to science class work (e.g. in the

following item “an important reason I do my school work is because I like to learn

new things” I substituted “science work” for “school work”). See the Appendix for

complete surveys.

Post-survey

The post-survey included two open-ended questions that were designed to

assess students’ attributions for winning and losing in the game. However, this

measure was unsuccessful, as students tended to respond with the rules for winning

and losing in the game. In the end, this question served as a manipulation check,

demonstrating that 87% of EPB students and 95% of Control students correctly

understood how winning worked in the game, in their respective conditions.1 There

were also 6 Likert-scale items created for this study to assess students’ attitudes and

motivations towards the game. Questions focused on intrinsic motivation, self-

efficacy, and judgment of learning.

1 Two primary coders coded 25% of the manipulation check data and achieved 90% agreement. Due to this high level of reliability, a single coder coded the remaining 75% of the manipulation check responses.

49

Interview

The interview data will not be part of the analysis in this dissertation. This will

be reserved for future analyses.

Log Data

A database logged many of students’ discrete choices in the game, creating a

sequential record of students’ actions. It tracked the topics students attempted, and all

learning resources used. It also logged the amount of time students spent on each of

these resources.

Several two-event sequences were coded in the log data to explore the

relationship between failure, success, and persistence. Our measures of persistence

included “fail-abandon” and “success-abandon”. A “fail-abandon” behavior occurred

when a student failed a game and then immediately moved to a different planet (topic).

When students did not fail-abandon, they stayed on the same game level after failure.

“Success-abandon” was coded similarly to “fail-abandon,” but it occurred after a

successful game.

Measures of productive persistence explored what students did when they

persisted on a level. A “fail-play-again” behavior occurred when a student failed a

game and then immediately played another game on the same level. A “fail-resource”

behavior occurred when a student failed a game and then immediately checked a

resource on the same level. “Success-play-again” and “success-resource” were coded

the same way, except they occurred after successful games.

In all of these measures, viewing “feedback” was counted as part of the game

play process. For example, sequences of fail-play-again and fail-feedback-play-again

were both coded as “fail-play-again” behaviors. This was done because students

checked the “feedback” fairly frequently after failure (about 50% of the time) and also

because it was a qualitatively “easy” choice to make. To check any of the other

resources after playing a game, students would have to return to the homepage before

clicking on the desired resource. The feedback option was offered to every student

immediately after they played a game, and it was displayed as a special button at the

50

bottom of the win/lose page. In other words, since “feedback” only required 1 click,

while every other resource required 2-4 clicks, this was a more inviting resource that,

we decided, did not require the same amount of effort that went into checking the

other resources. Therefore, “feedback” was not treated as a diagnostic choice.

51

CHAPTER 4: EFFECTS OF TREATMENT ON LEARNING AND

PERSISTENCE: A FOUR-PRONGED ANALYSIS

Overview of findings

This chapter explores the main hypotheses of the study. We originally

predicted that students in the EPB condition would learn more than those in the

Control condition. This hypothesis was not supported. However, we did find that

high-failing students in the Control group learned less than their low-failing

counterparts, while high-failing EPB students learned just as much as their low-failing

EPB counterparts. In search of a mechanism for these learning effects, we explored

differences in persistence after failure across low- and high-failing groups in each

condition. While not statistically significant, the high-failing Control students were

relatively more likely to abandon a game after failure and persist on a game after

success. Finally, abandonment after failure was a negative predictor of learning -- the

less students persisted after failure, the less they learned. To hone in on the specific

locale of these effects, we investigated these same questions as a within-subjects

analysis, where we compared students in high and low failure situations (rather than

high and low failing students). The findings were similar, though the learning

differences between high and low failure situations were not as sizeable. Finally, we

explored these same hypotheses in the context of high and low challenge situations,

when students encountered games for which they had low and high prior knowledge.

This analysis showed that the effects of the ego-protective buffer on learning were

really a response to failure vs. success rather than a response to high vs. low challenge.

Overall, it seems that the ego-protective buffer is most effective under conditions of

high failure, particularly for high-failing students. See Table 5 on page 69 for a

summary of the main findings.

52

Analysis logic

The main analyses in this chapter were driven by three central research

questions, which relate to the original hypotheses:

Question 1: Does provision of an ego-protective buffer affect learning from the

game?

It was hypothesized that the EPB condition would have higher overall gain

scores than the Control group.

Question 2: Does provision of an ego-protective buffer affect persistence after

failure or success?

It was hypothesized that the ego-protective buffer would allow students to

persist after failure, which would lead to a lower rate of abandonment after

failure. However, we predicted that the ego-protective buffer would not

affect students’ response to success. To control for the rate of

abandonment, we compared persistence after failure to persistence after

success across conditions.

Question 3: Does persistence after failure or success affect learning from the

game, and does this effect differ by condition?

It was hypothesized that persistence after failure would increase learning

gains, on the assumption that abandonment after failure indicates a lost

opportunity to learn. If persistence after failure has an impact on learning,

then students who do not persist after failure should learn less from the

game, regardless of condition.

To investigate the above questions, four different types of analyses were

carried out. Each analysis examines the effect of the ego-protective buffer on learning

behaviors and outcomes from slightly different angles.

Analysis 4.1: Treatment effects

The first analysis examines treatment effects using the student as the unit of

analysis. Students’ choices and behaviors across the full game were related to

53

their overall learning gains. These effects were explored across Control and

EPB conditions.

Analysis 4.2: Treatment effects for high and low failers

This second analysis, also on the subject level, examines whether the effects of

the ego-protective buffer differ based on individual characteristics of the

student. This analysis asks whether students who experience a high rate of

failure in the game differ from students who experience a low rate of failure in

the game and whether this difference interacts with condition. Given that the

ego-protective buffer was designed to protect students from the negative

motivational consequences of failure, the EPB may only affect those students

who experience a high rate of failure.

Analysis 4.3: Treatment effects for high and low failure situations

The third analysis examines the effect of failure as a within-subjects variable.

The unit of analysis here is the situation rather than the student. In Analysis 2,

we asked whether the treatment would affect high and low-failing students

differently. In this analysis, we ask whether the treatment would have

differential effects on students’ responses to high and low failure situations.

For example, do students show greater rates of abandonment for games where

they fail often compared to those where they do not? Another interesting

difference between this analysis and the prior analyses is that it makes a tighter

connection between actions taken on specific levels of the game and learning

outcomes on those levels (because it is possible to match items on the posttest

to each game individually).

Analysis 4.4: Treatment effects for high and low prior knowledge situations

A final analysis examines the question of persistence from a different angle.

Rather than investigate the effects of failure on persistence, this analysis

examines the effect of high challenge (as a situation-level variable) on

persistence. When students know little about the topic covered in a game, they

may be less likely to persist after failure because the game is too challenging

for them. In this analysis, high challenge situations were defined as games for

54

which students had low prior knowledge. Provision of an ego-protective

buffer may moderate the effect of challenge on in-game behavior by providing

students, prospectively, with a lower risk of the negative ramifications of

failure.

There are advantages to testing the research questions with several different

analyses. First, it allows us to separate real effects from “fluke” effects that may

appear when the data are sliced one way but not another. Second, it allows us to

investigate these effects when either the situation or the person is the unit of analysis,

to gain more precision in understanding the conditions under which the effects take

place. Third, it allows us to investigate the effects at different degrees of specificity.

Some effects are investigated on the broad scope, examining how behaviors across the

entire game relate to overall learning, while others are investigated more precisely,

tying specific behaviors on levels of the game to learning gains on those levels.

Analysis 4.1: Treatment effects

This first analysis focuses on the effect of treatment on learning gains, in-game

choices, and the relationship between in-game choices and learning outcomes. Before

addressing each of these pieces of the analysis, we explore whether students learned

from the game, regardless of condition.

All analyses in the dissertation include students who were in the main

classroom and the pullout study. Their behaviors looked quite similar and there were

no mean differences in behaviors across pullout and classroom students. Note that the

number of students in each analysis presented below may change. 143 students

completed the full two days of instruction, but only 130 of them had pre- and post-test

data. Analyses that describe students’ in-game behaviors only used the full set of 143

subjects, though in some cases, students had missing data for a certain behavioral

outcomes, so the N may drop. In other analyses, the unit of analysis is at the level of

the observation rather than the student, so the degrees of freedom could become larger

than N.

55

Learning Scores from Pre- to Posttest

The first two rows of Table 4.1 display pre- and posttest scores for the full

sample and split by condition. Because some students never visited a level of the

game, their pre- and posttest scores on the unvisited levels were not included in their

average pre- and posttest scores. Because students were given the choice of what to

learn, it only seems sensible to test them on what they attempted to learn.

Furthermore, this should tighten the relationship between in-game behaviors and

learning outcomes. The number of levels visited was equal across EPB and Control

groups (MControl = 6.88, SDControl = 0.38; MEPB = 6.77, SDEPB = 0.55), so this alteration to

the learning scores did not bias the analysis in favor of either condition2.

Pretest scores were similar across conditions, demonstrating equivalent prior

knowledge across the groups, t(128) = .34, p = .74. To test whether students learned

from the game, students’ average question scores were compared from pretest to

posttest. On a paired sample t-test, pre- and post-test scores differed significantly,

t(127) = 10.40, p < .001, d = . 48. On average, students’ scores increased from pre- to

posttest by 10 percent, a moderate improvement.

Table 4.1. Average scores on pretest, posttest, learning gains, and transfer items. Throughout, parenthetical values are standard errors of the group means.

All Control EPB Pretest 0.52 (.02) 0.51 (.03) 0.53 (.03) Posttest 0.63 (.02) 0.62 (.03) 0.64 (.03)

Gain Score 0.16 (.02) 0.15 (.02) 0.17 (.02) Near Transfer 0.19 (.02) 0.19 (.03) 0.19 (.03) Far Transfer 0.33 (.03) 0.30 (.04) 0.36 (.04)

Learning Gain Scores

To measure learning from the game, a gain score was derived from pre- and

posttest scores. Rather than a simple difference score, which disadvantages high pre- 2 Even when we do not alter the learning scores in this way, the results comparing pretest, posttest, and learning gain scores across conditions are the same. However, removing the scores for unvisited levels does tighten the connection between actions taken in the game and learning outcomes (i.e. most correlations between in-game behaviors and learning outcomes become slightly more significant).

56

test scorers, the difference score was divided by the possible gain or possible loss.

This gives all participants equal “room” to gain or lose from pre- to posttest. To

normalize the distribution of gain scores, the square root of the denominator was

taken. In cases where the difference score was 0, the gain score also became 0. Table

4.2 shows the formula applied to compute gain scores.

Table 4.2. Calculation of learning gain scores. Post-Pre Gain Score Formula

> 0 (Post – Pre) / √(1 – Pre) < 0 (Post – Pre) / √ Pre 0 0

4.1.a Question 1: Does provision of an ego-protective buffer affect learning from the

game?

The key hypothesis was that the ego-protective buffer would facilitate learning.

To test this prediction, we compared EPB and Control conditions on average learning

gains from pre- to posttest. The effect of condition on learning gain was not

significant, t(126) = .89, p = .37 (see Table 1).

After taking the posttest, students completed an additional test containing near

and far transfer items that were not part of the pretest. Transfer items focused on two

main concepts that ran throughout the game: probability and models of inheritance.

Near transfer items required application of these concepts to novel situations within

the domain of genetics. Far transfer items required application of these concepts to

novel situations beyond the genetics domain.

To test the effect of condition on near and far transfer items, a repeated

measures ANOVA crossed the within-subjects factor of transfer type (near vs. far)

with condition on test scores. There was no effect of condition, F(1, 139) = .52, p =

.47 and no interaction, F(1, 139) = 1.21, p = .27. However, there was a significant

effect of transfer type, F(1, 139) = 32.31, p < .001, demonstrating that students scored

lower on the near transfer items. Given that the near transfer items were designed to

be easier than the far transfer questions, this result suggests that the transfer items

57

were not assessing the appropriate constructs. In light of this, the transfer measures

will not be used as outcome measures in the remainder of the analyses.

4.1.b Question 2: Does provision of an ego-protective buffer affect persistence after

failure or success?

Another main hypothesis was that the EPB manipulation would discourage the

behavior of abandoning after failing a level of the game. To test this prediction, the

rate at which students abandoned a level after success was compared to the rate of

abandonment after failure across high and low failing students in both conditions. The

rate of fail-abandon was computed by dividing the total number of abandonments3

after failure by the total number of failures in the game. Dividing by the number of

fails ensures that any effects are due to abandonment rather than failure per se. To

compute the success-abandon rate, the number of abandonments after success was

divided by the total number of successes. Rates of success-abandon and fail-abandon

created a simple within-subjects comparison that controls for the effect of overall rate

of abandonment regardless of failure.

The frequency and time spent on various behaviors across each day of game

play were highly correlated (all r’s > .46), indicating that students behaved similarly in

both game sessions. Given this, all in-game behaviors such as the rate of fail-abandon

and the rate of success-abandon were collapsed across both days of game play in all

analyses.

To test the effect of condition on abandonment after success and failure, a

repeated measures ANOVA crossed the between-subjects factor of condition on the

within-subjects factor of success-abandon vs. fail-abandon. There was a marginal

main effect of success- vs. fail-abandon, F(1,129) = 3.07, p = .08, indicating higher

rates of abandonment after failure. There was no effect of condition, F(1,129) = .11, p

3 An abandonment occurred when a student played a game and then immediately moved to another level of the game, abandoning the prior level. An abandonment was coded when students either (a) played a game and then changed levels or (b) played a game, viewed feedback on that game, and then changed levels. Feedback was treated as part of failure because it occurred quite frequently after failure (approximately 50% of the time) and because it was too tempting of a resource choice to be diagnostic.

58

= .74. However, there was a significant interaction of condition by success- vs. fail-

abandon, F(1,129) = 6.00, p = .02, d = .2. As shown in Figure 1, and confirmed by

Tukey’s post hoc tests, the EPB students were equally likely to abandon after a

success or failure, while the Control students were far more likely to abandon after

failure than after success, p < .05. By this analysis, the hypothesis that the Control

condition would fail-abandon more was partially confirmed. The Control students did

not fail-abandon at a significantly lower rate than the EPB group, but the Control

students’ rate of fail-abandon was lower than their own rate of success-abandon. So,

the ego-protective buffer of the EPB condition did not simply reduce the incidence of

fail-abandon. Rather, it equalized the likelihood of abandonment after success and

failure. Figure 4.1 shows this pattern of results.

Figure 4.1. Rates of fail-abandon and success-abandon by condition.

4.1.c. Questions 3: Does persistence after failure or success affect learning from the

game, and does this effect differ by condition?

To examine the effect of fail-abandon and success-abandon on learning, simple

correlations with the gain scores were computed for the full sample and for each

59

condition (see Table 4.3). There were no significant correlations between success-

abandon and learning. However, there was a significant negative correlation between

fail-abandon and learning in the full sample. When broken out by condition, both

Control and EPB groups showed small, negative, non-significant correlations between

fail-abandon and learning gains. These results indicate that the more students

abandoned after failure, the less they learned from the game, regardless of condition.

This is a rather weak effect, but nonetheless it shows that students’ response to failure

had an impact on learning.

Table 4.3. Correlations between learning gains and rates of success- and fail-abandon, overall and split by condition. All Control EPB Success-abandon -.05 -.06 -.08 Fail-abandon -.18* -.14 -.23

* significant at p < .05

Analysis 4.1 Summary

Analysis 1 demonstrated that students across both conditions learned complex

genetics material by playing the game. However, there were no effects of treatment

on learning gains. An examination of in-game persistence behaviors revealed that

students in the Control condition were more likely to abandon a game topic after

failure than after success, while EPB students were equally likely to abandon after

success and failure. One interpretation of this result is that the EPB is acting in two

ways. First, it eliminates the avoidant reaction to failure. Second, an ego-protective

buffer also affects the prospective treatment of failure, whereby it reduces the

tendency (displayed by the Control group) to “play it safe” and stay in one’s comfort

zone, rather than venture out and risk failure on a new challenge. Persistence in the

face of failure has consequences for learning. The more students abandon a topic after

failure, the less they learn, whether or not they have access to an ego-protective buffer.

60

Analysis 2: Treatment effects for high and low failers

The ego-protective buffer of the EPB condition was designed to support

students in persisting after failure, so one potential hypothesis is that the EPB will

have stronger effects on students who experience more failure. Reciprocally, the EPB

may not provide a learning benefit for students who rarely fail. To test this prediction,

students were binned into high- or low-failers based on failure rates (number of fails

divided by number of games played). Categorizing people into the dichotomous

categories of high and low failers provides a simple way to conceptualize the effect of

failure on learning in each condition.

Groups were split on the median to ensure equal sample size in the high and

low failing groups. Both the median and mean failure rates were nearly identical

across conditions (MedianControl = .50, MeanControl = .52, SDControl = .29; MedianEPB = .50,

MeanEPB = .48, SDEPB = .29). On average, students failed 50% of the games they

played4. Figure 4.2 shows that the distribution of failure rates is relatively normal

across conditions, so there is no danger that the median split obscures the effects of a

non-normal distribution.

4 Recall that failure occurs when a student scores at or below 75% on a single game.

61

Figure 4.2. Histograms of failure rates by condition (failures divided by number of games played).

4.2.a. Question 1: Does provision of an ego-protective buffer affect learning

differentially for high and low-failing students?

To test this question, a factorial ANOVA crossed the factors of condition by

high/low failers to predict learning gain. There was no main effect of condition F(1,

126) = .34, p = .56, but there was a significant main effect of high- versus low-failers

F(1, 126) = 17.75, p < .001, demonstrating a lower gain score for the high-failers.

Most importantly, there was a significant interaction effect F(1, 126) = 5.86, p = .02, d

= .2. As shown in Figure 4.3, students in the Control condition who had a high rate of

failure learned far less than their low-failing counterparts. However, in the EPB

condition, high- and low-failers made the same learning gains, demonstrating the kind

of buffer effect the EPB was designed to maintain. Post-hoc comparisons using

Tukey’s HSD indicated that the gain score of high- and low-failers in the Control

conditions differed significantly, p < .05, but this was not the case in the EPB group.

62

Figure 4.3. Learning gain scores by condition by high- versus low-failers.

Similarly, when removing the binning, the rate of failure had a stronger

relationship with learning gains in the Control condition (r = -.50, p < .001) than in the

EPB condition (r = -.28, p = .03). In the Control condition, each additional failure was

associated with a larger decrement in gain score than in the EPB condition (see Figure

4.4). Taken together, the results of the ANOVA and correlations indicate that failure

had a stronger effect on learning in the Control condition than in the EPB condition.

This fits the overall hypothesis that providing students with a psychological buffer

protects them from the negative ramifications of failure that can interfere with

learning. However, it suggests a modification to the original hypothesis. The

protection offered by an EPB is most important for students who experience high rates

of failure.

63

Control EPB

Figure 4.4. Scatterplots of failure rate by learning gain for each condition.

4.2.b. Question 2: Does provision of an ego-protective buffer affect persistence after

failure or success differently for high and low-failers?

Given the learning differences between high and low-failers by condition, the

next analysis examined whether the rate of fail-abandon and success-abandon differed

across high- and low-failers within each condition. To test this, a repeated measures

ANOVA crossed the between-subjects factors of condition and high/low failers on the

within-subject factor of success- vs. fail-abandon. Once again, the interaction of

success- vs. fail-abandon by condition was significant, F(1,127) = 5.99, p = .02,

replicating the earlier finding that Control students were more likely to fail-abandon

than success-abandon, and EPB students were equally likely to abandon after either

game outcome. There was also a significant effect of high/low failers on

abandonment, F(1,127) = 16.59, p < .001, demonstrating that high-failers were more

likely to abandon. There was a marginal effect of success vs. fail-abandon, F(1,127) =

2.87, p = .09, which indicates higher rates of abandonment after failure than success.

There was a marginal interaction of condition by high/low failers, F(1,127) = 2.88, p =

.09, which indicates a larger difference in abandonment rates between high and low

64

failers in the EPB condition than the Control condition. There were no other

significant effects, all F’s < 2.0. The lack of a significant three-way interaction shows

that abandonment behaviors were not drastically different across low and high-failers

by condition. However, Figure 4.5 shows that, descriptively, the Control group

exhibited lower rates of success-abandon than fail-abandon, while EPB students were

equally likely to success- and fail-abandon. Control students who were failing

frequently tended to abandon a level they had failed and stayed on a level they could

pass rather than risk failure on a new level.

Figure 4.5. Abandonment after success and failure by condition and high- versus low-failers.

4.2.c. Question 3: Does persistence after failure and success affect learning from the

game, and does this effect differ by condition and high- and low-failer students?

To examine whether condition affects learning differently for high and low-

failers, correlations between fail-abandon and success-abandon rates and learning

gains were computed and listed in Table 4.4. None of the correlations is significant

when partitioned to this level.

65

Table 4.4. Correlations between the rates of success- and fail-abandon and learning gain scores, broken out by condition and high- and low-failers. Control EPB

Low Failure High

Failure Low Failure High

Failure Success-abandon -.05 -.01 .003 -.14 Fail-abandon -.04 .03 -.15 -.22

Analysis 4.2: Summary

The analysis of failure-rate as a person-level characteristic demonstrated that

without an ego-protective buffer, the Control students who failed at a high rate learned

less from the game than low failers. In contrast, high-failure students in the EPB

condition did not show a learning decrement relative to low failers. We suspected that

this difference in learning gains could have been caused by students’ differential

responses to failure across the two conditions. Analysis 1 demonstrated that, overall,

EPB students were just as likely to abandon a level after success and after a failure,

while Control students abandoned after failure at a higher rate than they abandoned

after success. These differences were even more pronounced for the high failure

Control group, though not significantly so. However, at this level of partitioning, fail-

abandon did not correlate significantly with learning. Thus, the chain of effects is that

a lack of an ego-protective buffer influences high-failure students to abandon

relatively often. The missing link in the chain is the effect of fail-abandon on learning.

When analyzing the larger sample this link is present -- students who fail-abandoned

learned less – which completes the full chain of inference, though not at the same level

of analyis.

Analysis 3: Treatment effects for high and low failure situations

The prior analyses suggest that the ego-protective buffer has a greater impact

on high-failing students. This next section asks whether the ego-protective buffer

makes a stronger impact in high-failing situations, across all students. For this

analysis, persistence and learning were compared across levels of the game where

66

students experienced high and low rates of failure regardless of whether students were

high- or low-failers.

For each student, each level of the game was classified as a high or low failure

situation based on a cut-off of 50% failure. Given that 50% failure was the median

split used in the previous analysis to categorize students as high- or low-failers, the

same cut-off of 50% failure was set to classify each level as a high or low failure

situation. In other words, levels on which students lost more than 50% of the games

were considered high-failure situations. Classifications of levels into high and low

failure situations were done independently for each student. The average behaviors

and learning gains were computed for each students’ high- and low-failure situations.

This analysis allows us to examine behavior and learning when students fail more

often than they succeed and when they succeed more often than they fail.

This type of analysis occurs on a more narrow scope than the former analyses;

it ties level-specific behaviors to level-specific learning outcomes. In the earlier

analyses, overall in-game behaviors were related to the overall gain scores. In this

analysis, actions on a level will be related to posttest questions for that level.

Not every student had both high and low rates of failure. However, the

number of students with low failure situations and the number of students with high

failure situations were equal across conditions (NHighControl = 69, NHighEPB= 69; NLowControl =

57, NLowEPB = 54), so splitting the data by high and low failure situations did not favor

one condition over another. Mixed model regressions will be used to analyze these

data. Mixed models can account for multiple observations of subjects, in unbalanced

designs with missing data, such as this one.

4.3.a. Question 1: Does provision of an ego-protective buffer affect learning from the

game differently in high and low failure situations?

A mixed model regression tested the effect of condition by high/low failure

situations on learning. The factors of failure situation (high vs. low) and condition

were entered in a factorial design with average learning gain score on high and low

failure levels as the outcome measure. The model uncovered a significant main effect

67

of failure situation, F(1, 221) = 8.59, p = .004 with a beta estimate of .07 for low

failure situations. This suggests that students learned more from low-failing

situations; students’ gain score in low-failing situations is .07 units, almost half a

standard deviation higher than their learning gain from high-failing situations. The

effect of condition and the interaction between condition and failure rate were not

significant, F’s < 1.5. However, descriptively, the results are in the same direction as

in analysis 4.2.a. As shown in Figure 4.6, the difference in learning outcomes between

high and low failure situations is greater in the Control condition.

Figure 4.6. Learning gains by high/low failure situations by condition.

4.3.b. Question 2: Does provision of an ego-protective buffer affect persistence after

failure or success differently in high and low failure situations?

To assess the effect of high/low failure situations on abandonment after

success and failure, a repeated measures ANOVA was run, crossing the factors of

condition and high/low failure situations on success- vs. fail-abandon. There was a

significant main effect of failure situation, F(1, 189) = 13.95, p < .001, indicating

higher overall abandonment on high failure levels. There was also a significant

interaction between success- vs. fail-abandon and high/low failure situation, F(1, 189)

68

= 15.62, p < = .001, demonstrating that students were more likely to abandon after

failure than success when the failure rate was high but they did the opposite when the

failure rate was low. Just as in the first analysis, there was a significant interaction of

success- vs. fail-abandon by condition, F(1, 189) = 5.74, p = .02, showing that the

Control condition was far less likely to abandon after success than after failure, while

the EPB condition was equally likely to do either. Most importantly, there was a

significant three-way interaction between success- vs. fail-abandon, high/low failure

situation, and condition, F(1, 189) = 6.34, p = .01. The pattern is clearly depicted in

Figure 7. When Control students were in high-failure situations, the rate of success-

abandon was much lower than the rate of fail-abandon, whereas for EPB students in

high-failure situations, these rates were equivalent. Moreover, rates of fail-abandon in

high failure situations were higher in the Control group than the EPB group, while

rates of success-abandon in high failure situations were lower in the Control group

than the EPB group, as confirmed by Tukey’s test, p’s < .05.

So, when students were failing less than 50% of the time, they were more

willing to “stick with” a level when they failed but were quick to leave when they

passed, regardless of the condition they were in. But when they were failing more

than 50% of the time, condition mattered. Students in the EPB group, who were

experiencing high failure on a level, were just as likely to give up on a level after

success as after failure. However, Control students in the same situation were far

more likely to give up after failure and “stick with” a level they could pass. These

findings dovetail nicely with the results from the person-level failure analysis, which

showed the same pattern of results.

69

Figure 4.7. Rates of abandonment after success and failure split by condition and high/low failure situations.

4.3.c Question 3: Does persistence after failure affect learning from the game, and

does this effect differ by condition? Do these effects differ across high and low failure

situations?

To test the effect of abandonment after failure on learning in Control and EPB

groups during high and low failure situations, a linear mixed model was run. The

factors of failure situation (high vs. low), condition, fail-abandon rate, and their

interactions were entered into a model predicting learning gain scores. There was a

marginal interaction between failure situation and rate of fail-abandon, F(1, 202) =

3.72, p = .06, indicating that fail-abandon had a stronger effect on learning in high

failure situations. However, no other effects were significant, F’s < 2.22. This

analysis suggests that the rate of fail-abandon does affect learning in high failure

situations. A second mixed model was run with the same predictors but with success-

abandon rates as the outcome measure. There were no significant effects, F’s < 2.50,

suggesting that success-abandon rates do not explain learning gains.

70

Analysis 4.3 Summary

An analysis at the within-subjects level did not find a significant interaction

between high/low failure situations and condition on learning outcomes. However, the

pattern of results was descriptively similar, with the Control group showing a greater

loss of learning in high failure situations compared to the EPB group. This suggests

that rate of failure is an important pre-condition for the ego-protective buffer to protect

learning, but the effect of failure has a stronger impact at the subject level (as shown in

Analysis 2) than the situation level. This analysis did find that in high failure

situations, the Control group was more likely to fail-abandon than success-abandon,

while the EPB group was equally likely to do both. These results are quite similar to

those uncovered in Analysis 2. Finally, there was a near-significant effect where fail-

abandon was a better predictor of learning in high-failing situations. This suggests

that persistence after failure is particularly important with respect to learning, when

students are failing more than 50% of the time.

Analysis 4.4: Treatment effects for high and low prior knowledge situations

A final analysis examined the effect of the ego-protective buffer in challenging

situations. Perhaps the effects of the ego-protective buffer are not specific to failure

situations but can be generalized to any situation of great difficulty. We hypothesized

that the ego-protective buffer could encourage persistence in high challenge situations.

A ‘high challenge’ situation occurred when a child played a game for which he had a

low pretest score. Our assumption was that students would feel challenged when they

encountered a game topic they knew little about.

In this within-subjects analysis, each student’s set of game levels was

classified as a high or low prior knowledge level based on pre-test scores. Once again,

we chose a 50% criterion. Levels where students scored higher than 50% on the level-

specific pretest item were classified as high prior knowledge levels. Likewise, a score

of 50% or below on the pretest item classified a level as low prior knowledge. In

71

other words, high and low prior knowledge situations were defined as levels of the

game where a given student knew more or less than half of the material.

Not every student had both high and low prior knowledge levels. However,

the number of students with high and low prior knowledge levels was equal across

conditions (NHighControl = 57, NHighEPB= 60; NLowControl = 65, NLowEPB = 63). Mixed model

regressions will be used for this analysis, to account for missing data.

4.4.a. Question 1: Does provision of an ego-protective buffer affect learning

differently in high and low prior knowledge situations?

To test the effect of condition by prior knowledge within subjects on learning,

a mixed model regression was run. Prior knowledge (high or low) and condition and

their interaction were entered into the model to predict learning gain. Figure 4.8

shows the pattern of results. There was a large main effect of prior knowledge, F(1,

241) = 30.89, p < .001 with a beta estimate of .17 for low prior knowledge. There

were no other significant effects, F’s < 1. This suggests that the gain score is favoring

improvement on levels where students began with low prior knowledge. Moreover,

there is no interaction with condition. This finding suggests that the ego-protective

buffer acts on students’ response to failure rather than the general experience of

difficulty, such as encountering a game that is challenging.

72

Figure 4.8. Learning gain by condition and high/low prior knowledge situations.

4.4.b. Question 2: Does provision of an ego-protective buffer affect persistence after

failure or success differently for high and low prior knowledge situations?

To test the effect of treatment by prior knowledge on abandonment after

success and failure, a repeated measures ANOVA was run. Condition, success- vs.

fail-abandon, and prior knowledge (high vs. low) were entered into the model to

predict rates of abandonment. There was a main effect of prior knowledge, F(1, 178)

= 8.17, p = .005, d = .2, demonstrating higher overall rates of abandonment in the low

prior knowledge situations. There was a marginal effect of game outcome, F(1, 178)

= 2.70, p = .10, indicating slightly higher rates of abandonment after failure than after

success. There was also a condition by game outcome effect, F(1, 178) = 8.41, p =

.004, demonstrating that as before, the Control group was more likely to abandon after

failure than success, while the EPB group was equally likely to abandon after success

and failure. Finally, there was a small but significant interaction of game outcome by

prior knowledge, F(1, 178) = 3.84, p = .05, d = .14, revealing higher rates of

abandonment after failure as compared to success in the low prior knowledge group.

The high prior knowledge group was equally likely to engage in abandonment after

73

failure and success. The three-way interaction was not significant. Descriptively, the

pattern is the same as before. Figure 4.9 shows that students in the Control condition

experiencing high challenge, which in this case is the low prior knowledge group, the

discrepancy between the rates of fail-abandon and success-abandon is quite large in

comparison to the other three groups.

Figure 4.9. Rates of abandonment after success and failure split by condition and high/low prior knowledge situations.

4.4.c. Questions 3: Does persistence after failure affect learning from the game, and

does this effect differ by condition or situations of high and low prior knowledge?

The factors of condition, prior knowledge (high vs. low), and fail-abandon

rates were crossed in a mixed model regression to predict learning gain scores. There

was a main effect of prior knowledge, F(1, 220) = 10.64, p = .001, such that situations

of low prior knowledge resulted in greater learning gains, with a beta estimate of .11,

more than one third of a standard deviation. There was also a main effect of fail-

abandon rate, F(1, 219) = 6.97, p = .009, with a beta estimate of -.22, about three

quarters of a standard deviation, suggesting that in all groups, fail-abandon was a

74

significant predictor of learning. There were no other main effects or interactions, all

F’s < 2.6. As seen in Analysis 1, this model confirms that fail-abandon behaviors

impact the learning of all students.

To test the effects of success-abandon rates by condition and prior knowledge

on learning gains, an identical mixed model was run, except success-abandon rates

were entered into the model in lieu of fail-abandon rates. Only the main effect of prior

knowledge was significant, F(1, 183) = 18.72, p < .001, with a beta estimate on low

prior knowledge of .23, about three quarters of a standard deviation. All other main

effects and interactions were non-significant, F’s < 1.39. As before, success-abandon

is not explaining a significant amount of the variance in learning gain scores.

Analysis 4.4 Summary

In contrast to the interaction effects between failure rates within and across

subjects, a challenging situation, conceptualized as low prior knowledge, did not affect

learning gains differentially by condition. Consistent with prior analyses, there was a

descriptive (though non-significant) trend for the Control group in low prior

knowledge situations to abandon more frequently after failure than success. In

addition, fail-abandon rates did predict learning gains, and this did not interact with

condition or prior knowledge. This reiterates the findings from Analysis 1, where fail-

abandon has a negative effect on learning for all students.

75

Table 4.5. Summary of findings from four data analyses, addressing three research questions.

Discussion

Taken together, these four sets of analyses define the conditions under which

the ego-protective buffer does and does not affect persistence and learning. Moreover,

using each analysis as a different test of the main hypotheses helps to identify the

robust findings and weed out those that might simply appear by a lucky partitioning of

the data.

With respect to our three central research questions, some of them are more

clearly answered by these four sets of analyses. The first question asked whether

provision of an ego-protective buffer affects learning and under what conditions.

There were no general effects of treatment on learning, however, high-failing students

in the Control group learned less than their low-failing counterparts. Descriptively,

high-failing situations were also associated with lower learning gains in the Control

group, though this finding was not statistically significant. This suggests that the

76

effect of failure by treatment on learning is stronger on the person level than the

situation level. Low prior knowledge situations, on the other hand, did not interact at

all with condition to affect learning. This suggests that the effect of the EPB on

learning is moderated largely by failure rather than the general experience of

difficulty.

The second research question focused on the behavioral effects of the ego-

protective buffer under various conditions. Here, the effects were fairly consistent

across analyses, suggesting that this finding is robust. Students in the Control group

tended to abandon a level of the game at a higher rate after failure than after a success,

while students in the EPB group were just as likely to persist after failure and success.

This effect was magnified in high-failing individuals, high-failing situations, and low

prior knowledge situations, though this pattern was only significant in failure

situations. Overall, the ego-protective buffer encourages persistence under times of

difficulty, particularly when subjects experience a high rate of situational failure.

Finally, we asked whether behaviors of persistence after success and failure

impacted learning and how this effect might be moderated by conditions of failure and

challenge. On the whole, students who did not persist after failure (by failing and then

abandoning) learned less from playing the game, regardless of condition or failure

rates.

While none of these analyses tell a perfect story by themselves, taken together,

they tell a clear story (see Table 4.5 for summary of findings). Persistence after failure

is critical for learning; students who do not persist learn less from the game. An ego-

protective buffer enhances persistence after failure relative to success, particularly in

high failure situations. Perhaps because of this5, students with an ego-protective

buffer who experience high failure learn just as much as their low-failing counterparts,

while high-failing students who do not have access to an ego-protective buffer learn

far less than their low-failing counterparts. Interestingly, students in the Control

5 It should be noted that we have not found a clear relationship between the difference in persistence behaviors across conditions and learning outcomes. While persistence after failure is related to learning, the EPB and Control groups differ on persistence after failure only as it relates to persistence after success.

77

condition were also more likely to make safer choices after success, by choosing to

play games they knew they can win. These effects also occur, but to a lesser degree,

in situations of high failure.

Why are the effects significant in some analyses and not others? One

possibility is that condition by failure has a small effect on learning. So partitioning

the data in one way leads to significant results, while partitioning the data another way

creates consistent, but not quite significant results. Moreover, because the data are

sometimes at the person-level and sometimes at the level of the observation, different

statistical tests were run. Because some statistical tests were more powerful than

others, they might create discrepancies in statistical significance across analyses. A

third possibility is that the measure of learning is changing from one analysis to the

next. On the person-level analyses, learning gain is based on the average question

score across all test items. On the within-subjects analyses, learning gain is based the

average question score across all items on a certain type of level (e.g., high/low failure

situation). This slight change in measures may have affected the findings.

So far, we have examined the effects of treatment on persistence and

persistence on learning gains. We have shown that an EPB can enhance persistence

under conditions of high failure relative to success and that persistence after failure

can influence learning. However, persistence after failure (as measured here – fail-

abandon) had a very small effect on learning. It only explains about 4% of the

variance in learning gains. It is possible that the type of persistence or more complex

behavioral patterns in the game will have a greater impact on the learning of high-

failing students. The next section will explore productive persistence and other in-

game behaviors, in hopes of uncovering other possible causes of the learning effects.

The subsequent analyses take place at the level of Analysis 2 (between-subjects failure

by condition), since it was most productive in explaining learning outcomes.

78

CHAPTER 5: BEHAVIORS BEFORE AND AFTER SUCCESS AND FAILURE

Overview of findings

In this chapter, we examine how behaviors both before and after success and

failure predict learning, and how they differ by condition, high/low failers, and their

interaction. Of the behaviors following success and failure, fail-abandon was the most

predictive, so persistence itself had a large effect on learning. How students persisted

had less of an impact on learning in this study. However, some interesting behavioral

patterns emerged from these analyses. For instance, students were more likely to

check resources after failure than after success, particularly amongst low failers. Also,

students in the control group were more likely to play another game on the same level

after experiencing success than after failure. One interpretation of this result is that

the Control students were more risk-averse than the EPB students, and so they were

more likely to stay on a level they knew they could win. However, this success-play-

again behavior was not significantly correlated with learning.

Analyses of behaviors preceding game play show that high failers were more

likely to switch levels before game play. One interpretation is that these students were

jumping from level to level, in search of a game they could win. However, we also

found that level-change-fail (i.e., switch to a new level of the overall game and then

fail) is more common than level-change-success, suggesting that switching levels may

have caused these students to become high failers. Findings must be interpreted with

caution, since we cannot determine the direction of the causal arrows. Future analyses

could explore other measures of preparation for game play that would more precisely

titrate these effects.

Analyses

The prior chapter examined how the experimental treatment affected

persistence and how persistence affected learning. This chapter looks beyond simple

persistence to explore other types of behaviors that precede and follow failure. The

purposes of this section are three-fold. First, we aim to better define the term

79

“productive persistence” by understanding what kinds of responses to failure are

associated with enhanced learning. Second, we explore the actions preceding success

and failure, in an attempt to further explain why the high-failers in the control group

learned less in the game. Third, all analyses are broken out by condition, high/low

failers, and their interaction, to explore the effect of each of these factors on

differences in failure-related behaviors and learning. Because rate of failure as a

between-subjects variable had the most significant effects on learning, we will confine

our analyses to treatment by high/low failers only, and forgo the analyses of high/low

failure situations and prior knowledge.

Analysis 5.1: Productive Persistence

This section explores what students did when they chose to persist. After

students played a game on a level and chose to persist with that level, they could

persist in two ways. Students could either play another game at the same level, or they

could check a relevant resource such as a reading, solution, explanation, or preview.

This section examines whether one of these types of persistence is more productive by

exploring which one is associated with greater learning gains. We expected that

checking a resource after failure would constitute more productive persistence. Rather

than solving the problem through trial and error by playing repeatedly, the students

would be attempting to remediate their faulty understandings through explanations of

the content. If the high-failers in the Control group engaged in less productive

persistence, it could further explain their relatively lower learning gains.

The following analyses will examine two central questions. The first is

whether the rate of each type of persistence differs by condition, by high/low failers,

and their interaction. The second question we test is whether each type of persistence

has consequences for learning, and whether this differs by condition, high/low failers,

and their interaction.

Once again, the measure of each type of persistence was calculated as a rate.

For instance, the rate of checking a relevant resource after failure (“fail-resource”) was

calculated by dividing the total number of fails-followed-by-resource by the total

80

number of fails. Likewise, the rate of persisting by simply playing another game on

the same level (“fail-play-again”) was computed by dividing the total number of fail-

play-again sequences by the total number of fails. We compared these rates to the

“success-resource” and “success-play-again” rates, to effectively control for any

differences in the overall rate of accessing resources or playing continuous games.

Table 5.1 displays the rates for each of these persistence measures, broken out by

condition and high/low failers.

Table 5.1. Rates of various types of responses to success and failure, split by condition and high/low failers. Low Failers High Failers Control EPB Control EPB Success-abandon .39 (.04) .38 (.04) .41 (.04) .60 (.04) Fail-abandon .45 (.05) .38 (.05) .59 (.05) .56 (.05) Success-play-again .58 (.04) .56 (.04) .52 (.04) .34 (.04) Fail-play-again .24 (.04) .32 (.04) .23 (.04) .26 (.04) Success-resource .03 (.02) .05 (.02) .07 (.02) .06 (.02) Fail-resource .32 (.05) .30 (.05) .19 (.05) .19 (.05)

5.1.a Rates of persistence after success and failure, broken out by condition and

high/low failers

To examine differences in resource use after game play, a repeated measures

ANOVA crossed the between-subjects factors of condition and high/low failers,

repeated on the within-subjects factor of success-resource vs. fail-resource. There was

a large main effect of game success- vs. fail-resource, F(1, 127) = 65.34, p < .001,

indicating that students were more likely to check a resource after failure than after

success. There was also a significant interaction of success- vs. fail-resource by

high/low failers, F(1, 127) = 8.83, p = .004, demonstrating that while both high and

low-failers were more likely to check a resource after failure than after success, the

discrepancy between rates was much smaller in the high-failer group. In other words,

high and low-failer students were equally likely to check a resource after success, but

the low-failer group was far more likely to do so after failure. There was also a

81

marginal main effect of failure rate, F(1, 127) = 3.50, p = .06, indicating that low

failers were more likely to check a resource after failure, in general. There were no

other significant effects, F’s < 1. Notably, there were no significant three-way

interactions that might account for the high-failer Control group’s unique learning

outcomes. The bottom two rows of Table 1 display rates of success- and fail-resource.

To explore differences in the rate of play-again persistence, a repeated

measures ANOVA crossed the between-subjects factors of condition and high/low

failers on the repeated, within-subjects factor of success-play-again vs. fail-play-again.

There was a large main effect of success- vs. fail-play-again, F(1, 127) = 125.87, p <

.001, showing that students were twice as likely to play again after success than after

failure. There was also a main effect of high/low failers, F(1, 127) = 7.51, p = .007,

demonstrating that low failers were more likely to play again, regardless of outcome.

There was a significant interaction of high/low failers by success- vs. fail-play-again,

F(1, 127) = 5.70, p = .02, revealing that low failers were more likely to play again

after success. Perhaps this is why they are low failers, because they kept playing

games when they won. There was also an interaction of condition by game outcome,

F(1, 127) = 12.59, p = .001, demonstrating that while both conditions were equally

likely to play again after failure, the Control condition was more likely to play again

after success. This is another indication that the Control students were risk averse,

preferring to play games when they knew they could succeed. Once again, there was

no significant three-way interaction that could contribute to the learning effects for the

high-failing Control students. The middle rows of Table 1 show the rates of game

play following success and failure.

5.1.b Correlations with learning gains, split out by condition, high/low failers, and

their interaction

To examine the relationship between learning and types of persistence, we

computed correlations between learning gains and persistence (play-again and

resource-use) partitioned by success and failure. None of the correlations were

significant (all r’s < .13), suggesting that neither one of these types of persistent

82

behaviors is instrumental for learning in the game. So far, only persistence after

failure (fail-abandon) has been significantly associated with learning outcomes.

5.1 Summary

These analyses showed that students were more likely to check a resource after

failure than after success, particularly amongst low-failer students. Common sense

says that this should be a productive behavior, since checking a resource is probably

an attempt to remediate understanding. However, failing and checking a resource was

not correlated with learning in any group of students. With respect to the play-again

type of persistence, students were more likely to play-again after a successful game,

especially in the Control condition, suggesting that these students were more risk

averse, avoiding potential failure situations. However, since none of these behavioral

measures correlates significantly with learning, we were unable to identify what

counts as productive persistence based on this analysis. So far, we have only been

able to demonstrate that persistence (fail-abandon) matters but not productive

persistence. Also, we did not find that any of these behaviors were related to the

poorer learning outcomes of the high-failer Control group.

Interestingly, these results suggest an alternative interpretation of the findings,

namely that behaviors before/after failure and success cause high or low failure rates

in the game. Since low failers were more likely to play again after success and were

also more likely to view a resource after failure, this suggests a different direction for

the causal arrow. Perhaps engaging in these behaviors causes students to fail less,

placing them in the category of low-failing students. So rather than interpreting these

behaviors as a response to failure, we could interpret them as a cause of failure. If this

were true, then one would expect these behaviors to have a significant association with

learning gains. But given that these behaviors are not correlated with learning,

perhaps they are really a response to failure rather than a cause.

83

Analysis 5.2: Behaviors preceding success and failure

This section examines the events preceding success or failure in the game.

There were no strong hypotheses. But, because the high-failing control group seemed

more sensitive to the effects of failure, it is plausible that this effect may be partially

caused by what students chose to do before game play. To examine this, we compared

conditions and high/low failers on their rates of various behaviors preceding success

and failure, and we also examined the relationship between these behaviors and

learning outcomes.

Events preceding success and failure were broken into three broad categories.

Game play could be preceded by another game played on the same level, checking of

a relevant resource, or a change of levels. Each of these behaviors was computed as a

rate against the total number of successes or failures. For example, the rate of “level-

change-success” is the total number of times students changed levels and then

immediately played a successful game divided by the total number of successful

games. Likewise, the rate of “game-success” is the rate that students played a game,

and then played another successful game in the same level divided by the total number

of successes. Finally, the rate of “resource-success” is the total number of successful

games preceded by relevant resource checks divided by the total number of successes.

Similar rates were computed for failure. Table 5.2 shows the rates of each type of

behavior preceding success and failure, split by condition and high/low failers.

Table 5.2. Rates of actions preceding success and failure, split by condition and high/low failers.

Low Failers High Failers Control EPB Control EPB

Level-change-success .26 (.04) .21 (.04) .31 (.04) .42 (.05) Level-change-fail .32 (.04) .31 (.04) .50 (.04) .53 (.05)

Game-success .54 (.04) .54 (.04) .43 (.04) .39 (.04)

Game-fail .34 (.04) .35 (.04) .24 (.04) .24 (.04)

Resource-success .19 (.04) .26 (.04) .26 (.04) .20 (.04) Resource-fail .34 (.05) .34 (.04) .25 (.04) .23 (.05)

84

5.2.a Test of mean differences by condition, by high/low failers, and their interaction

To test whether one condition or group of high- and low-failers differed in the

rate of level-changes before failure or success, a repeated measures ANOVA was

carried out. The between-subjects factors of condition and high/low failers were

crossed on the repeated, within-subjects measure of level-change-success vs. level-

change-fail. There was a significant effect of high/low failers, F (1, 128) = 16.97, p <

.001, indicating that high-failers were more likely to switch levels prior to playing a

game. One interpretation is that high-failers were switching levels in search of a game

they could pass. Another interpretation is that high-failers took more risks by jumping

straight into game play on new levels, which in turn, causes them to become high-

failers. A significant main effect of success/failure, F (1, 128) = 34.03, p < .001,

demonstrated that failure was more commonly preceded by a level-change. This

makes sense given that students were probably more likely to be successful on a level

they had already worked on in some capacity (either played or examined a resource

beforehand). There was a marginal high/low failer by success/failure interaction, F (1,

128) = 3.45, p = .07, indicating that high failers were more likely to level-change-fail

than level-change-succeed, while low failers were equally likely to do either. Perhaps

high failers took more risks by jumping into harder or less familiar levels of the game,

or perhaps they were engaging in less methodical choices, simply jumping around

from level to level. There were no other significant effects, nor interactions by

condition, F’s < 2.44. The top two rows of Table 2 display the means for level-

change-success and level-change-fail.

To test the effects of continuous game play, a repeated measures ANOVA

crossed the between-subjects factors of condition and hi/lo failers on the within-

subjects factor of game-success vs. game-fail. The repeated measures ANOVA

yielded a large significant main effect of game-success vs. game-fail, F (1, 128) =

81.79, p < .001, showing that successes were more often preceded by game play than

failures. There was also a significant main effect of high/low failers, F (1, 128) =

85

13.20, p < .001, indicating that low failers were more likely to play two consecutive

games in a row. All other effects were insignificant, F’s < 1. See Table 2 for means.

To test the effect of resource use before game play, a repeated measures

ANOVA crossed the between-subjects factors of condition by high/low failers on the

within-subjects factor of resource-success vs. resource-fail. There was a significant

effect of resource-success vs. resource-fail, F(1, 128) = 13.54, p < .001, indicating that

students were more likely to check a resource before failure. There was also a

significant effect of high/low failers by resource-success/resource-fail, F(1, 128) =

8.34, p = .005, suggesting that while high failers were equally likely to check a

resource before success and failure, low failers were far more likely to check a

resource before failure. See Table 2 for means of resource-success and resource-fail.

5.2.b. Correlations with learning

To examine whether the events preceding success and failure affected learning

gains, correlations between learning gains and behaviors are shown in Table 5.3.

Correlations are split out by condition, high/low failers, and their interaction. There

are only three significant correlations. In the Control group, level-change-fail (i.e.,

switch into a level, immediately play a game and fail at that game) is moderately

negatively correlated with learning while it has no relationship in the EPB condition.

A much stronger and more interesting correlation is that high failers in the Control

group have a strong negative correlation between level-change-success and learning

gains. This correlation may reflect a tendency for the high-failer Control group to

leave hard levels and seek out levels they have already mastered, simply to experience

success. This connects with the earlier finding that high-failer Control students tended

to stay on a level where they have experienced success. Both are cases of

conservativism or avoidance of challenge. Finally, there is a positive correlation

between resource-and-then-fail and learning gains in the high-failer control group

only. There is also a positive correlation between resource-success and learning gains

in the same group, so checking resources before game play appears to have helped the

high-failer Control students.

86

Table 5.3. Correlations between learning gains and events preceding success and failure.

*significant at p < .05, ** p < .01

5.2 Summary

This section examined how behaviors preceding success and failure differed by

condition, high/low failers, and their interaction. With respect to level-change

behaviors, one interesting finding is that when students changed levels immediately

before playing a game, they were more likely to fail than succeed. This indicates that

when students did not prepare for a game (either by checking a resource or playing a

game on that level), they were more likely to fail. Also, high-failers were more likely

than low-failers to switch levels and play a game. Again, this suggests that switching

levels before game play leads to failure. However, another interpretation is that high-

failers were desperate to find a level they could pass, so they jumped around from

level to level, playing games along the way, in search of a game they could win.

Unfortunately, the simple measure of transitions from level-change to failure or

success will not allow us to differentiate between these two explanations, and since the

data are correlational, we cannot know which factor is the causal agent.

Analyses of behaviors preceding success/failure revealed that when students

played two games in a row, on the same level, the second game was more often a

success than a failure. Also, low-failers were more likely than high-failers to play two

Low Failers High Failers

All Control EPB Low

Failers High

Failers Control EPB Control EPB Level-change-success -.11 -.22 .01 .07 -.14 .04 .08 -.49** -.01 Level-change-fail -.16 -.26* -.03 -.02 -.01 .01 -.06 -.19 .11 Game-success .17 .22 .11 .04 .14 .05 .02 .22 .19 Game-fail .15 .15 .15 .14 -.06 .11 .19 -.16 .04 Resource – success -.04 .03 -.14 -.12 .03 -.11 -.09 .36 -.20 Resource – fail .03 .14 -.10 -.09 .07 -.09 -.11 .34* -.17

87

consecutive games in a row. Checking a resource before failure occurred more often

than checking a resource before success, especially in the low-failing group. This

suggests that the low-failer students were employing more metacognition. They were

more likely to check a resource when they realized they had low knowledge, but

unfortunately, because they had low knowledge and the resources were difficult, they

were still likely to fail at the game, despite their resource use. However, it is also

possible that students who were low-failers actually failed less because they engaged

in less risky behaviors by continually playing games on the same level (rather than

trying something new).

The relationship between learning and behaviors preceding failure and success

showed some differences by condition and high/low failers. Interestingly, level-

change-fail is negatively correlated with learning, in the Control group. This suggests

that changing levels before failure, which occurs quite frequently, is only instrumental

for learning in the Control condition. Also, level-change-success is negatively

correlated with learning gains, while resource-fail is positively correlated with

learning, in the high-failing Control group only. It is unclear why these behaviors are

only significantly correlated with learning in the high-failing Control group.

Discussion

Other behaviors surrounding success and failure (beyond simple persistence)

did not reveal additional factors that might have contributed to the decrement in

learning gains for the high-failer Control group. The close examination of types of

persistence after failure yielded some interesting results, but it did not help to define

the term “productive persistence” since none of the persistent behaviors was

significantly related to learning outcomes. The exploration of behaviors preceding

success and failure also did not clearly explain the learning effects. However, both

analyses provided some interesting data describing patterns of behaviors around

success and failure. For instance, students were more likely to check a resource before

and after failure than before or after success, especially amongst low-failers. Students

were also more likely to fail than succeed immediately after changing levels.

88

However, students were more likely to succeed when playing another game on the

same level.

There were some interesting findings with respect to condition differences. In

chapter four, we found that Control students, particularly those experiencing high

failure, were more likely to stay on a level they could pass. To corroborate this

finding, analyses in this chapter showed that students were more likely to play again

after experiencing success, particularly in the Control condition. This could occur

because students in the Control condition were more risk-averse than the EPB

students. Without the ego-protective buffer, Control students might have been

protecting their own egos by continuing to play games they knew they could win.

Because this is correlational data, we can only speculate about the cause

behind the relationships in the data. For instance, high-failers were more likely to

switch levels before playing a game. One interpretation is that high failure caused

level-switching; high-failing students were clicking from level to level, in search of a

game they could win. An alternative interpretation which reverses this causal arrow is

that level-switching itself caused high failure because students who were level-

switching were not practicing or checking resources before playing games. Of course,

level-switching and failure could be in a reciprocal relationship as well. In any case,

the interpretations here are provisional and cannot be duly tested without more

sophisticated analyses that take into account whether students return to prior levels

where they have had success.

Moreover, more accurate behavioral measures of what students do prior to

game play are needed to secure some of these claims. For instance, a student could

switch levels and immediately play a game, which we would interpret as a lack of

preparation for game play. However, this student might have checked several

resources on this level much earlier in the sequence of “clicks”, but our measures

would not have captured this. Also, these measures blur the line between behaviors

following game play and behaviors preceding game play. For instance, a student may

play a game, check a resource, and then play another game, all on the same level. It is

unclear whether the resource check was in response to the first game played or in

89

preparation for the next game. Examining what students do before they attempt the

first round on a game level would help eliminate some problems of interpretation here.

There are countless ways to code for behaviors around success and failure; in this

dissertation, we have chosen only one way to assess them (simple transitions between

two events).

In the next chapter, we examine how persistence behaviors can be predicted by

pre-existing differences like motivation and prior achievement in science class. Given

the potential overlap in events preceding and following game play (and the risk of

double-counting), we will constrain our focus to events that occur after success and

failure. Moreover, behaviors after game play are more closely tied to learning

outcomes (especially fail-abandon).

90

CHAPTER 6: PRE-EXISTING INDIVIDUAL DIFFERENCES AS

PREDICTORS OF IN-GAME BEHAVIORS AND LEARNING OUTCOMES

Overview of Findings

This section examines how well pre-intervention measures of individual

differences, including motivation and prior learning, predict learning outcomes and

persistence. It examines these effects across the full sample of subjects, by condition,

by high/low failers, and by their interaction. The results show that pre-existing

differences in global, motivational traits do not predict learning gains, while more

proximal measures of in-game persistence (i.e. fail-abandon) do. This suggests that

examining measures of in-game behaviors does have some predictive power. Another

interesting marginal finding is that prior achievement and failure in the game are better

predictors of learning in the Control condition than in the EPB condition. This

suggests that the EPB manipulation somehow “levels the playing field” for high and

low achievers and also reduces the impact of failure on learning. Finally, failure is

more predictive of both prior achievement and prior learning for the high-failing

group, compared to the low-failing group. This suggests that failure is perhaps more

instrumental for learning amongst students who experience a great deal of it.

Introduction

Many educational outcomes can be largely explained by individual differences

such as prior achievement, prior knowledge, gender, motivation, and so on. Because

these differences exist prior to the educational treatment, this leaves very little room

for the effect of an educational intervention. Educators hope that changeable, state-

like characteristics can have an effect on learning because they offer suggestions for

educational reform. With this in mind, this section examines how pre-existing

individual differences predict in-game behaviors and learning outcomes and whether

more proximal measures, like in-game behaviors add any predictive power. We have

chosen to focus on two individual differences that are particularly critical, given the

context of the work: motivation and prior learning. Because the intervention is

91

designed to affect students’ situational motivation (in response to failure), it is

important to examine whether other, more stable and global aspects of motivation

have an effect on student behavior and learning. For instance, rather than focus on

attributions given in response to failure, the motivational measures collected prior to

treatment assess general motivational traits such as interest in science class and beliefs

about inborn intelligence. One of the questions we pose in this section is how well

pre-existing individual differences in motivational traits predict in-game behaviors and

learning outcomes. If specific behaviors in the game (like fail-abandon) add some

predictive power above and beyond pre-existing differences, then they may prove

useful as measures of persistence that have an impact on learning processes and

outcomes.

Another interesting question is whether the effect of individual differences

differs by our factors of interest. In keeping with the structure of the prior chapter, we

will first conduct our analysis with the full sample of data. Next, we will compare

differences across condition, then across high and low failers. Finally, we will

examine the interaction of condition by high and low failers. Within each of these

analyses, we will examine the relationships between (1) pre-existing differences and

behaviors in response to failure, and (2) pre-existing differences and learning

outcomes.

Given the tremendous number of statistical tests that were conducted for these

analyses, the alpha level for significance has been set to p < .01. For the most part,

correlations that do not meet this criterion will not be discussed as noteworthy effects.

This should help avoid the problem of spurious correlations and lower the chances of a

Type 1 error, plus it renders the number of significant correlations to a tractable level.

We will begin with an analysis of the motivation measures and then move on

to measures of prior learning. The analyses in this section are quite exploratory, and

many of the effects do not have ready explanations relevant to the thesis of the

dissertation. However, they do provide a fuller description of the factors affecting

game play and learning outcomes.

92

Measures of motivation and prior learning

There were seven measures of motivation: self-regulated learning6, mastery

orientation, performance-approach orientation, performance-avoid orientation,

interest, self-efficacy, and fixed intelligence. Each measure was geared towards

motivation in science class (e.g. “I like to show others how smart I am in science

class”).

There were several measures of learning, taken both before and after the

treatment. Measures of learning taken prior to the treatment are used to predict

learning outcomes, measured after treatment. There were two measures of learning

prior to the treatment: science class grades and pretest scores. Science class grades

serve as a measure of prior achievement in science class, while the pretest is a more

specific measure of content knowledge covered in the game. The two post-measures

include failure rate in the game itself, which is the most proximal measure of learning

from the game, and learning gain scores from pre- to posttest.

Hypotheses

There were no specific hypotheses for this section. However, we did make

some educated guesses as to what students with high and low levels of each

motivational type would do, with respect to failure-response behaviors:

• Students high in self-regulated learning or mastery motivation should be more

likely to use resources after failure because they are motivated to learn and

master the content.

• Students with a performance-approach orientation should be more likely to

play again after success because they want to show what they know. They

may also be more likely to change levels after failure, because they primarily

want to demonstrate their ability to succeed.

• Performance-avoid students should be more likely to fail-abandon because

they tend to avoid situations that show their lack of competence.

6 Self-regulated learning is both a motivational and metacognitive measure.

93

• Students with high fixed intelligence beliefs should behave similarly to the

performance-approach and performance-avoid students, because both

orientations are based in a belief of static intelligence that mitigates attempts to

improve.

• Students high in self-efficacy should be less likely to fail-abandon since they

believe they can succeed.

• Depending on the situation, one condition’s individual differences may

correlate more strongly with behaviors and learning. For instance, if the

treatment is decreasing the level of fail-abandon in all EPB students, then

individual differences should have little effect on their rate of fail-abandon.

However, individual differences should have a greater impact in the Control

condition, where the treatment is not pushing all students to engage in a

particular behavior.

• The effects of motivation on behavior and learning may be stronger in the

high-failer group, because motivational resources are often recruited in times

of difficulty.

• Because they are more proximal measures, in-game behaviors should explain

some unique variance in learning gain scores, above and beyond pre-existing

individual differences.

Reliability analyses for motivation and learning measures

Reliability analyses were carried out on each of the seven motivation scales.

Cronbach’s alpha was moderately good on most measures (α ranged from .51 to .84,

Table 6.1). The least reliable scale is the one for fixed intelligence, which makes

sense, given that it has the fewest items. On the whole, the items of each scale are

fairly internally consistent.

94

Table 6.1. Reliability analyses for each motivation scale

Scale Number of

Items Cronbach's

Alpha Self-regulated learning 14 0.84 Mastery orientation 4 0.72 Performance-approach orientation 4 0.68 Performance-avoid 4 0.61 Interest 4 0.78 Self-efficacy 4 0.69 Fixed Intelligence 3 0.51

Because of a lack of item-specific data, it was impossible to do reliability

analyses for science class grades, failure rates, and learning gain scores. However,

both pre- and posttest measures were reliable (αpretest = 0.79, αposttest = 0.83).

Motivation analyses

6.1: Analyses of the full data sample

6.1.a. Average levels of pre-existing motivation

To get an idea for students’ pre-existing motivation, Table 6.2 displays the

means and standard errors for each motivational measure. The self-regulated learning

scale was measured on a different scale (1-7) from the others (1-5). A score of 1

signified that a student “disagreed a lot” with the statement, while a score of 5 or 7

indicated they “agreed a lot.” A rating of 4 (in the case of self-regulated learning) or a

rating of 3 for the other scales indicates the midpoint of the scale or “neutral”

agreement with the statement. On average, students were just above “neutral” on all

scales except for fixed intelligence, where they were closer to a rating of 2 or

“disagree a little.”

95

Table 6.2. Average ratings on motivation measures given prior to treatment Mean SE Self-regulated Learning (1-7) 4.34 0.08 Mastery (1-5) 3.18 0.07 Performance approach (1-5) 3.28 0.07 Performance avoid (1-5) 3.22 0.07 Interest (1-5) 3.28 0.07 Self-efficacy (1-5) 3.51 0.06 Fixed intelligence (1-5) 2.33 0.08

6.1.b. Relationship between individual differences in motivation and pre- and post-

learning measures

How does pre-existing motivation predict learning from the game? How does

it predict prior knowledge and achievement? To answer these questions, Table 6.3

displays correlations between motivation measures and learning measures. Only two

motivational measures correlate with any measure of pre- or post-treatment learning.

Fixed intelligence is negatively correlated with pretest scores and positively correlated

with failure rate. Individuals who more heavily endorsed a model of fixed intelligence

knew less content knowledge before playing the game and experienced more failure

during the game. As Dweck’s (2000) work suggests, beliefs of fixed intelligence can

have many negative impacts on learning behaviors and outcomes because they

discourage students from making an effort.

Table 6.3. Correlations between motivation measures, prior achievement, and learning gains Pre-treatment learning Post-treatment learning

Science grade Pretest Failure rate Learning

Gain Self-regulated learning -.18 .14 -.15 .07

Mastery .09 .02 .00 .00 Performance approach -.02 -.07 .08 .00

Performance avoid -.11 -.16 .16 .01 Interest .02 .03 -.08 -.07 Self-efficacy .21 .22 -.26* .07 Fixed intelligence -.22 -.24* .32* -.04 * significance at p < .01

96

Self-efficacy was a negative predictor of failure. Students who were high in

self-efficacy for science also experienced less failure in the game. Self-efficacy is

often a good predictor of achievement and learning, showing that what students

believe they are capable of doing contributes to learning and performance. Other

motivational measures like self-regulated learning, performance and mastery

orientations, and interest were not significantly associated with any learning measures.

Interestingly, none of the motivation measures were significantly correlated

with learning gain scores, our favored measure of learning from the game. This

suggests that pre-existing motivational measures leave room for in-game measures of

motivation (like persistence) to explain some of the variance in learning gains.

6.1.c. Relationship between pre-existing motivation and in-game behaviors in

response to failure

Even though motivation measures do not predict learning from the game, they

may predict in-game behaviors, some of which do predict learning outcomes. Table 4

in the Appendix displays the correlations between the various persistence type

behaviors that could follow success or failure in the game: abandonment, game play,

or resource use. The only correlation that meets the alpha criterion of .01 is between

self-regulated learning (SRL) and fail-resource (r = .23, p < .01). Students with high

SRL were more likely to check a resource after failure. SRL is also positively

correlated with success-resource but this is not a significant correlation. This suggests

that students who are good self-regulated learners were likely to check resources after

playing a game and especially after a failed game. Perhaps students who were able to

regulate their own learning realized the need to remediate their knowledge after failing

a game.

Interestingly, none of these motivation measures predicted the fail-abandon

behavior. Recall that fail-abandon is negatively correlated with learning gains. The

functional data of fail-abandon may serve as a better learning-relevant motivation

measure for this game-situation than survey data.

97

6.2 Condition Differences

6.2.a. Average levels of pre-existing motivation

To determine whether conditions differed on pre-existing motivation measures,

a multivariate ANOVA tested the effect of condition on all 7 motivation measures:

self-regulated learning, mastery, performance-approach, performance-avoid, interest,

self-efficacy, and fixed intelligence. There was no effect of condition, F(7, 128) = .20,

p = .99, indicating their equivalence at the start of the study.

6.2.b. Relationship between individual differences in motivation and pre- and post-

treatment learning measures

Correlations with pre- and post-treatment learning measures indicate that self-

regulated learning, self-efficacy, and fixed intelligence are all more strongly correlated

in the Control condition (see Table 6.5). These patterns are similar to the ones found

in the full sample of data, but they appear to be driven by the Control condition. Self-

efficacy is a positive predictor of prior learning measures and a negative predictor of

failure in the game, while fixed intelligence predicts in the opposite direction.

Contrary to the analyses from 6.1.a, self-regulated learning is a positive predictor of

science grade and pretest and a negative predictor of failure. These correlations are

only significant in the Control group. However, because these differences between

treatments occur on measures taken prior to the treatment (i.e science grade, pretest),

the correlations between these motivation measures and post-treatment learning

outcomes become suspect. It suggests that these are pre-existing differences in the

way motivation relates to learning/knowledge in the Control group, rather than

differences that occurred as a result of the treatment.

98

Table 6.5. Correlations between measures of motivation and pre- and post-treatment learning measures, split by condition. Pre-treatment learning Post-treatment learning Science Grade Pretest Failure Learning Gain Control EPB Control EPB Control EPB Control EPB

Self-regulated Learning .42* -.02 .34* -.04 -.31 .00 .26 -.15

Mastery .28 -.09 .14 -.11 -.06 .06 .16 -.21

Performance approach -.004 -.02 -.08 -.05 .14 .02 -.10 .12

Performance avoid -.08 -.14 -.17 -.15 .17 .14 .09 -.10

Interest .18 -.12 .19 -.10 -.15 -.02 .15 -.32 Self-efficacy .51* -.08 .39* .04 -.43* -.04 .26 -.19

Fixed intelligence -.32* -.11 -.36* -.13 .41* .23 -.15 .09

* significance at p < .01

6.2.c. Relationship between pre-existing motivation and persistence behaviors

To examine possible differences in how motivation predicts behaviors across

conditions, Table 6.6 in the Appendix demonstrates correlations between motivational

measures and in-game behaviors in response to failure, broken out by condition.

There are two significant correlations that meet the .01 criterion. The first is a small,

negative correlation between performance avoid motivation and the success-play-

again (r = -.37, p < .01) behavior in the EPB condition, while the correlation in the

Control group is close to zero. We predicted that performance avoid would be

negatively correlated with fail-play-again because students would avoid negative

performance situations. However, this was not the case.

Another interesting correlation occurs in the Control condition between

mastery motivation and fail-play-again (r = -.33, p < .01). Students who frequently

failed and then played another game on the same level had lower mastery scores. One

would expect mastery-oriented students, who strive to understand the content and

master the skills necessary for the task, to attempt to rectify any misunderstandings

that led to a failed game. So rather than play again after failure, they might be more

likely to view a resource, and in fact the correlation between fail-resource and mastery

99

is positive (though not significant), corroborating this hypothesis. It is unclear why

this correlation only appears in the Control condition.

6.3 High and low failers

6.3.a. Average levels of pre-existing motivation

To assess group differences in motivation, a multivariate ANOVA tested the

effect of high/low failers on all 7 motivation measures: self-regulated learning,

mastery, performance-approach, performance-avoid, interest, self-efficacy, and fixed

intelligence. There was a significant overall effect of between-subjects failure rate,

F(7, 122) = 3.37, p = .003. Separate univariate ANOVAs revealed that the low-failers

had higher self-efficacy, F (1, 128) = 9.64, p = .002 and endorsed ideas of fixed

intelligence to a lesser degree, F (1, 128) = 10.58, p = .001. Low-failers were

marginally less performance-avoid-oriented, F (1, 128) = 3.35, p = .07, and marginally

higher on self-regulated learning, F (1, 128) = 3.00, p = .09. There were no

differences between students’ endorsement of mastery motivation, performance-

approach motivation, or interest, F’s < 1.42. We predicted that the low-failers would

be higher in most motivation measures in general, so these results are not surprising.

Table 6.7 shows means and standard errors on each motivation measure, for low and

high failers.

Table 6.7. Average ratings on motivation measures, split by high and low learners Low Failers High Failers Mean SE Mean SE Self-regulated Learning 4.46 0.10 4.21 0.11 Mastery 3.20 0.09 3.17 0.09 Performance approach 3.22 0.11 3.35 0.10 Performance avoid 3.09* 0.11 3.35 0.09 Interest 3.37 0.09 3.20 0.10 Self-efficacy 3.69* 0.09 3.33 0.08 Fixed intelligence 2.09* 0.10 2.57 0.11

* significance at p < .05, low-failers vs. high-failers.

100

6.3.b. Relationship between individual differences in motivation and pre and post

learning measures

Once groups were broken out by high- and low-failers, there were no

significant correlations between pre-existing motivational differences and measures of

learning, either pre- or post-treatment (see Table 6.8 in appendix). This suggests that

the majority of the correlations that occurred in the full data sample are picking up

differences between high and low-failers, which essentially controls for the rate of

failure in these analyses.

6.3.c. Relationship between pre-existing motivation and in-game behaviors in

response to failure

Similarly, there were no significant correlations between motivational

measures and post-game behaviors such as fail-abandon, within either the high- or

low-failers (see Table 6.9 in appendix). Once again, the variance between high and

low failing students may be explaining the correlations between motivation measures

and persistence behaviors seen in the Control and EPB groups, so once that variance is

removed by splitting students into high and low failers, the correlations disappear.

This suggests that the motivation measures are really driven by failure; once failure is

controlled for, measures of pre-existing motivation no longer have an impact on

learning and behavior in the game.

6.4. Condition differences and their interaction with high/low failers

In this section we explore the possibility of interactions between condition and

high/low failers in their relationship between motivation and in-game behaviors and

learning measures.

6.4.a. Average levels of pre-existing motivation

To assess pre-existing differences in motivation across groups, a multivariate

ANOVA tested the effect of condition, high/low failers, and their interaction on the 7

motivation measures. As before, the effect of between-subjects failure rate was

101

significant, F (7, 126) = 3.39, p = .002, but condition effects and interactions were not,

F’s < 1.04. So high-failers had different levels of motivations from low-failers, but

high-failers in the Control condition did not look different from high-failers in the

EPB condition prior to the intervention.

6.4.b. Relationship between individual differences in motivation and pre and post

learning measures

There are two significant relationships between learning measures and

motivation measures, when the data are split out by condition and high/low failers.

The low-failer Control group shows a positive correlation between mastery motivation

and science grade (r = .45, p < .01). Students who have a mastery orientation and fail

less also have higher science grades, in the Control condition only. The high-failer

EPB group shows a negative correlation between interest and learning gains (r = -.48,

p < .01). Students who were more interested in the science content but failed a great

deal actually learned less in the EPB condition. There are no obvious explanations for

these effects. See Table 6.10 in appendix for all correlations.

6.4.c. Relationship between pre-existing motivation and in-game behaviors in

response to failure

Table 6.11 (in appendix) displays correlations that examine the relationship

between pre-existing motivation and behaviors in response to failure split by condition

and by high/low failers. Performance-avoid motivation has a moderate, negative

correlation with success-play-again, but only in the high-failing EPB group (r = -.55, p

< .01). This correlation was seen before in the EPB group as a whole, but this finding

suggests that it was driven by the high-failers. Perhaps performance-avoid-oriented

individuals avoid all opportunities for performance, even after a success. This may be

exaggerated in the high-failing EPB condition because motivation comes to bear in

situations of difficulty (high failure) and when there are no other forces pushing

behavior in one way (i.e. the Control “treatment” may be pushing everyone to engage

in performance-avoid behaviors). There is also a negative correlation between interest

102

and fail-play-again in the high-failer Control group (r = -.46, p < .01). This is

counterintuitive because interest should encourage students to continue engaging with

the subject of their interest.

Summary of Motivation Analyses

This analysis revealed that our most robust measure of learning from the game

– learning gain score – was not well-predicted by popular motivation measures or pre-

existing individual differences in motivation as measured by several scales. One

implication of this finding is that educational interventions can focus on changing

situation-specific, in-game behaviors (like persistence) which have an impact on

learning, rather than attempt to change global motivational traits (like interest).

Furthermore, none of the motivational measures were strong predictors of the

fail-abandon behavior which is important for learning from failure. This suggests that

in-game measures of behavior have the capability of explaining interesting

motivational phenomena, such as persistence after failure, that are not captured in

many popular motivational scales. Though perhaps another motivation scale, for

instance, one focused more on attributions, would have shown a stronger relationship

to persistence after failure in the game.

Measures of pre-existing motivation did make some predictions about

behaviors in response to failure. For instance, students who are good self-regulated

learners were more likely to check a resource after failure, presumably to remediate

their understanding. Self-regulated learning encouraged a healthy response to failure

in the game.

Somewhat troublingly, motivation measures were stronger predictors of prior

knowledge, prior achievement, and failure in the Control condition than in the EPB

group. This suggests that there were some pre-existing differences across conditions

in how initial motivation affected their prior learning. This casts some doubt on the

interpretation of any between-condition differences in the relationship between

motivation measures and in-game behaviors and post-treatment learning. These

103

differences may not have been caused by the treatment because they might have

existed prior to the treatment.

Many of the relationships between motivation and prior learning and between

motivation and in-game behaviors disappeared once the sample was split into high-

and low-failers. This suggests that these relationships were really driven by failure

itself (and motivation certainly does correlate with failure). So, failure is a significant

event that has a strong relationship to motivation and behavior, as we shall see in the

next section on prior learning.

Comparison of conditions and high/low failers by condition revealed a few

sporadic significant relationships between in-game behaviors and motivation

measures. However, none of these findings were particularly revealing with respect to

the main foci of the study, nor were they easily explained by motivational theory. It is

possible that due to the enormous number of statistical tests performed the

significance of these correlations was spurious.

Prior learning analyses

Having examined the relationship between pre-existing motivation and

learning outcomes and in-game behaviors, we now turn to explore the association with

prior learning. Once again, we want to determine whether in-game behaviors and

learning can be predicted by prior learning, and whether in-game behaviors have

predictive power for explaining learning outcomes.

There were two major measures of prior learning. The first is students’ end-of-

semester science class grades, given by their classroom teacher (since all students had

the same teacher, we expected science grades to be comparable across subjects).

Another, more proximal measure is pretest score; the pretest was given the day before

the treatment and directly assessed the skills and concepts covered in the game.

104

6.5: Analyses of the full data sample

6.5.a. Average levels of prior learning

To test the differences in prior learning measures between condition, high/low

failers, and their interaction, a separate two-way ANOVA was conducted for each

outcome measure: science grade and pretest score. Each ANOVA showed main

effects of high/low-failers, demonstrating higher science grade and scores for the low-

failers. There was no main effect of condition and no interaction. Results are

displayed in Table 6.12.

Table 6.12. Means and SEs of prior achievement and prior knowledge measures, split by condition and high/low failers.

Low Failers High Failers

Control EPB Control EPB Significant

Effect

Science Grade

92.39 (2.35) 91.47 (2.31) 75.20 (2.16) 71.85 (2.28)

Main effect of failure, p < .001

Pretest 0.66 (0.03) 0.67 (0.03) 0.38 (0.03) 0.37 (0.03)

Main effect of failure, p < .001

6.5.b. Relationship between pre- and post-treatment learning measures

The correlation matrix in Table 6.13 shows that pre and post-treatment

learning measures are closely related. For instance, pretest scores have a large,

negative correlation with failure, suggesting that prior knowledge is explaining a good

deal of the variance in failure rates across individuals. This also means that prior

knowledge is an excellent predictor of performance in the game.

Interestingly, learning gains are not strongly correlated with other measures,

particularly prior learning measures like science grade and pretest. Part of this is by

design, since the learning gain score was supposed to remove the relationship between

pretest and posttest so that it would measure pure learning gain and not differences in

post-test that could be explained by prior knowledge.

105

Table 6.13. Correlation matrix relating pre- and post-treatment learning measures

Science Grade Pretest Failure

Pretest .55* Failure -.62* -.81* Learning Gain .24* .18 -.39*

6.5.c. Relationship between prior learning and in-game behaviors in response to

failure

How does prior learning relate to persistence behaviors in the game? Table

6.14 shows correlations between persistence behaviors in the game and measures of

pre-treatment learning. For comparison’s sake, it also includes measures of post-

treatment learning. Both success and fail-abandon are negatively correlated with

pretest. One interpretation of this result is that students with low prior knowledge

were more likely to abandon a topic. An alternative interpretation is that students who

abandoned in the game were less persistent during learning in general, which has

caused them to learn less from prior instruction. Success and fail-abandon are both

positively correlated with failure. This could mean that students who abandoned were

more likely to fail in the game, or alternatively, students were more likely to abandon

when they experienced more failure.

Another finding is that success-play-again is positively correlated with science

grade and pretest and negatively correlated with failure. Fail-play-again is also

positively correlated with pretest and negatively correlated with failure (though not

significantly so). This suggests that individuals with high prior knowledge or low

failure in the game were more likely to continuously play games. Of course, the same

alternative interpretation applies as before. Students who were likely to persist by

trying again may have learned more from prior instruction.

Correlations with resource use are less clear. There is one significant

correlation between fail-resource and failure rate. Students who frequently checked a

resource after failing also failed less often. If students were learning from the

resources, then we would expect this to decrease their rate of failure.

106

It is interesting that many of these behaviors were predicted by pretest and that

many of these behaviors predicted failure, but the only behavior that predicted

learning gains is fail-abandon (at α = .05, since this was an a priori hypothesis), which

suggests that persistence after failure is critical for learning.

Table 6.14. Correlations between persistence behaviors and pre- and post-treatment measures of learning Pre-treatment Learning Post-treatment Learning Science Grade Pretest Failure Learning Gain Success-abandon -.15 -.24* .29* -.05 Success-play-again .22 .30* -.34* .09 Success-resource -.20 -.19 .15 -.12 Fail-abandon -.17 -.35* .35* -.18+ Fail-play-again .00 .28* -.13 .08 Fail-resource .17 .13 -.25* .13

* significant at p < .01; + significant at p < .05

6.6 Condition Effects

6.6.a. Relationship between pre- and post-treatment learning measures

One interesting difference in the relationship between pre- and post-treatment

learning measures is that in the EPB condition, both science grade and failure are less

predictive of learning gains (see Table 6.15). This suggests that there is something

about the ego-protective buffer intervention that “levels the playing field”, or divorces

the relationship between prior achievement and learning gains. Likewise, failure is

less consequential for learning gains, suggesting that the EPB really does “buffer” the

impact of failure.

107

Table 6.15. Correlations matrix of pre- and post-treatment learning measures, split by condition

Science Grade Pretest Failure

Pretest .57* Failure -.65* -.79* Control Learning Gain .37* .18 -.50* Pretest .54* Failure -.60* -.83* EPB Learning Gain .12 .17 -.28

* significant at p < .01

Regression analyses were run to follow up on these condition differences.

Learning gains were regressed on science grade, condition, and their interaction. As

shown in Table 6.16, the EPB condition and science grade were both significant

positive predictors, while the interaction term was a marginal, negative predictor. This

suggests that when science grade and the interaction between science grade and

condition are controlled for, the EPB condition has a much higher learning gain score.

Also, the “slope” of the relationship between science grade and learning gain is much

less steep in the EPB condition. However, this model only explains 6.4% of the

variance in learning gains. Because the interaction effect is only marginally

significant, the differences between conditions in the relationship between prior

achievement and learning gains are descriptive only.

Table 6.16. Regression results testing condition, science grade, and their interaction. (EPB dummy coded as 1 and Control as 0.)

B Std. Error t Sig. (Constant) -0.23 0.12 -1.93 0.06

EPB 0.31 0.16 1.95 0.05 Science Grade 0.01 0.00 3.24 0.00

EPB * Science Grade 0.00 0.00 -1.85 0.07

Table 6.17 displays the regression results for condition, failure rate, and their

interaction on learning gains. Failure rate is a significant negative predictor of

learning gain, while there is a marginal interaction, showing that failure rate has less

of an impact on learning in the EPB condition. This model explains 16% of the

108

variance in learning gain scores. Once again, these results should be taken with a

grain of salt, since the interaction of failure by condition is only marginally significant.

Table 6.17. Regression results testing condition, failure rate, and their interaction. (EPB dummy coded as 1 and Control as 0.)

B Std. Error t Sig. (Constant) 0.31 0.04 7.83 0.00 Treatment -0.06 0.06 -1.16 0.25 Failure rate -0.32 0.07 -4.67 0.00 Trtmnt * Failure rate 0.17 0.10 1.76 0.08

6.6.c. Relationship between prior learning and in-game behaviors in response to

failure

Correlations between prior learning and behaviors in response to failure show

stronger relationships in the EPB condition (see Table 6.18). Most behaviors in the

EPB condition are predicted by pretest scores and failure rates. For instance, both

success and fail-play-again are positively correlated with pretest, suggesting that high-

pretest scorers were more likely to play games continuously. Conversely, fail-

abandon was negatively correlated with pretest, suggesting that students with low

prior knowledge were more likely to abandon failed levels of the game. The same

relationships occurred with failure rates (though in the opposite direction). Both fail-

abandon and success-abandon were positively correlated with failure in the EPB

condition, which suggests that high-failing EPB students were more likely to abandon.

It is unclear why these behaviors are better predicted in the EPB condition. Finally,

one interesting relationship is that science grade negatively predicts success-resource

in the Control condition only.

109

Table 6.18. Correlations between learning measures and behaviors preceding success and failure Science Grade Pretest Failure Learning Gain

Control EPB Control EPB Control EPB Control EPB Success-abandon -.11 -.19 -.19 -.31 .19 .42* -.06 -.08 Success-play-again .23 .21 .18 .44* -.24 -.47* .10 .13 Success-resource -.37* -.05 .01 -.32 .18 .12 -.14 -.13 Fail-abandon -.10 -.25 -.24 -.46* .25 .47* -.14 -.23 Fail-play-again -.10 .09 .18 .37* .01 -.27 .05 .10 Fail-resource .17 .18 .11 .15 -.25 -.25 .11 .16

* significant at p < .01

6.7 High and low failers

6.7.a. Relationship between pre- and post-learning measures

An interesting difference between high- and low-failers is that for low-failers,

pretest is more closely related to science grades and learning gains, while for high-

failers, failure is more predictive of learning gains and science grades (see Table 6.19).

So, for students who experienced a great amount of failure in the game, that failure is

related to measures of prior learning but it also predicts future learning. Whereas, for

low-failers, failure did not have as large of an impact on prior and future learning. For

low-failers, pretest is the best predictor of learning gain, while for high-failers, failure

rate is most predictive of learning gain. Once again, failure is an important event for

students who experience a great deal of it.

110

Table 6.19. Correlation matrix between pre- and post-learning measures, split by high and low failers.

Science Grade Pretest Failure

Pretest .38* Failure -.15 -.42* Low Failers Learning Gain -.18 -.29 -.12 Pretest .21 Failure -.39* -.68* High Failers Learning Gain .20 .08 -.27

6.7.b. Relationship between prior learning and in-game behaviors in response to

failure

The only significant correlation here is between pretest and fail-play-again in

the low failers group (r = .37, p < .01). Low-failers with high pretest scores were

more likely to fail-play-again. Perhaps low-failers with high prior knowledge were

more likely to believe that they could succeed after failure if they tried again, whereas

students who experienced more failure were less likely to believe they could succeed.

See Table 6.20 in the appendix for the full table of correlations.

6.8. Condition by high/low failers

6.8.a. Relationship between pre- and post-treatment learning measures

There are two interesting relationships that emerged from this analysis. As

shown in Table 6.21, science grade is a strong negative predictor of failure in the high-

failer Control group. This suggests that high-failer Control students were following

the same pattern in the game as they do in science class (i.e. failure in science class

translates to failure in the game), while low-failers and to a lesser degree, high-failer

EPB students, were more likely to stray from that path. As usual, pretest is a strong

negative predictor of failure in most groups except for the low-failer Control group.

For them, pretest is a strong negative predictor of learning gain. This means that

students with low pretest scores in the low-failer Control group are making large gains

from pre- to post-test. There is no clear explanation for this effect.

111

Table 6.21. Correlation matrix of pre- and post-learning measures split by condition and high/low failers

Science Grade

Pretest Failure

Pretest .35 Failure -.19 -.31 Control Learning Gain

-.18 -.49* -.21

Pretest .40 Failure -.11 -.50*

Low Failers

EPB Learning Gain

-.21 -.08 -.02

Pretest .20 Failure -.42* -.61* Control Learning Gain

.30 -.09 -.13

Pretest .22 Failure -.34 -.76*

High Failers

EPB Learning Gain

.18 .21 -.45

* significant at p < .01

6.8.b. Relationship between prior learning and in-game behaviors in response to

failure

The one significant correlation appears between fail-play-again and pretest

scores in the low-failer EPB group (r = .51, p < .01). Students with more prior

knowledge about the content covered in the game were more likely to play again after

failure, particularly in the low-failer EPB condition. These correlations are more

attenuated in the other three groups. It is unclear why this correlation is stronger in the

low-failing EPB condition. See Table 6.22 in the appendix for the full table of

correlations.

Summary of Prior Learning Analyses

Some interesting new findings appeared when learning measures given prior to

the treatment are related to in-game behaviors and post-learning measures. First, prior

achievement is predictive of learning gains in the Control condition only. This

suggests that the EPB intervention reduces the relationship between pre-existing

learning differences and future learning outcomes. This is an exciting finding, given

112

that learning outcomes are usually very well-predicted by pre-existing differences that

are difficult to change. However, some measures of prior learning (pretest) were more

predictive of behaviors made in response to success and failure in the EPB condition.

It is unclear why measures of prior learning in the EPB condition do not predict future

learning but do predict in-game behaviors (like fail-abandon).

It also interesting to note that among the high-failers, failure in the game is a

strong predictor of both past and future learning. This suggests that failure is a critical

event for students who experience large amounts of it. These findings dovetail nicely

with the main finding in chapter 4, that high-failing students in the control condition

learn less than their low-failing counterparts.

Finally, as shown before, fail-abandon is the only persistence-related behavior

that predicts learning across the full sample of subjects. This means that persistence

after failure is critical, not simply persistence in general. The type of persistence,

whether consulting resources or playing another game, is less important for learning.

Discussion

There are many interesting findings from this chapter; we will discuss the

findings that are most meaningful with respect to the central foci of the research. One

question we posed in this section is how well in-game behaviors predict performance

and whether they add some predictive power, above and beyond measures of pre-

existing motivation and prior knowledge. We found that simple persistence after

failure (as embodied by the fail-abandon behavior) was the best predictor of learning

gains, when compared to other persistence type behaviors (e.g. success-abandon, fail-

resource, fail-play-again, etc.). Moreover, fail-abandon is a better predictor of

learning gains than more global, motivational measures. In fact, none of the

motivational scales were correlated with learning gains, nor did they predict the fail-

abandon behavior. This suggests that fail-abandon could serve as a behavioral

measure of persistence, and that this type of behavioral measure is more proximal to

learning than more global measures of motivational traits or perhaps more survey-

based measures of motivation. However, since fail-abandon is only weakly correlated

113

with learning and is less strongly correlated than other measures of prior learning,

such as science grade and failure in the game itself, fail-abandon is not the best

predictor of learning gains.

With respect to condition effects, the most interesting findings are that prior

achievement and failure in the game are marginally better predictors of learning in the

Control condition than in the EPB condition. This suggests that learning in the EPB

group is divorced from prior achievement; in a sense, the EPB manipulation has

“leveled the playing field” for high and low-achieving students. Also, learning in the

EPB condition is less sensitive to failure in the game; EPB students are more resistant

to the effects of failure on learning.

High-failers are extremely sensitive to failure. For high-failers, failure in the

game correlates with both prior achievement and prior knowledge. For high-failers,

failure is a meaningful event that, may consistently affect their learning in the past and

the present. For low-failers, prior knowledge is a critical measure that predicts both

prior achievement and failure in the game. What low-failers know predicts what they

learned before and how well they perform in the game. Given these results, it makes

sense that provision of an ego-protective buffer is more meaningful in the high-failing

group. For them, failure is most critical for learning.

Because these findings are based on correlational data, it is impossible to make

causal conclusions. We can speculate about the directions of the causal arrows, but

our interpretations must be taken with a grain of salt, because we cannot know the

cause of two variables that are associated with one another.

114

Appendix to Chapter 6

* significance at p < .01 in all tables below

Table 6.4. Correlations between motivation measures and persistent behaviors in the game

Success-abandon

Success-play-again

Success-resource

Fail-abandon

Fail-play-again

Fail-resource

Self-Regulated Learning

-.01 -.05 .16 -.15 -.11 .23*

Mastery .01 -.04 .10 -.06 -.14 .16

Performance Approach

.11 -.15 .11 .05 -.06 -.01

Performance Avoid

.17 -.22 .15 .22 -.12 -.13

Interest -.07 .01 .16 .03 -.17 .10

Self-efficacy -.01 -.01 .07 -.17 -.06 .22

Fixed Intelligence

.12 -.10 -.06 .17 -.03 -.15

Table 6.6. Correlations between motivation measures and persistent behaviors in the game, split by condition

Success-abandon

Success-play-again

Success-resource

Fail-abandon

Fail-play-again

Fail-resource

Ctl EPB Ctl EPB Ctl EPB Ctl EPB Ctl EPB Ctl EPB

SRL

-.05 .03 -.01 -.10 .14 .18 -.12 -.17 -.13 -.10 .21 .28 Mastery .07 -.06 -.12 .04 .14 .05 .07 -.20 -.33* .04 .16 .17

Perf. Approach -.01 .22 -.01 -.29 .05 .16 .07 .04 -.01 -.11 -.06 .06 Perf. Avoid .05 .29 -.08 -.37* .10 .20 .21 .24 .04 -.30 -.23 .02 Interest .04 -.15 -.08 .07 .11 .20 .13 -.07 -.27 -.07 .06 .14 Self-efficacy -.04 .01 .06 -.09 -.06 .20 -.10 -.25 -.13 .00 .19 .26 Fixed Intell. .08 .14 -.11 -.05 .10 -.21 .15 .21 .07 -.15 -.19 -.09

115

Table 6.8. Correlations between motivation measures and pre and post learning measures, broken out by high and low failers

Pre-treatment Learning Post-treatment Learning Science Grade Pretest Failure Learning

Low Failers

High Failers

Low Failers

High Failers

Low Failers

High Failers

Low Failers

High Failers

Self-regulated Learning .16 .10 .01 .12 .01 -.10 .02 .04

Mastery .28 .02 .05 -.03 -.07 .10 .13 -.16 Performance-approach .12 -.02 .06 -.08 .15 -.03 .07 -.02

Performance-avoid -.23 .10 -.18 .00 .24 -.10 .02 .10

Interest .21 -.16 .04 -.13 -.08 .07 .08 -.30 Self-efficacy .25 -.05 .17 -.06 -.13 .04 .02 -.08 Fixed Intelligence -.14 -.03 -.13 -.06 .09 .22 .12 -.02

Table 6.9. Correlations between motivation measures and persistent behaviors, split by high and low failers

SRL Mastery Perf.

Approach Perf. Avoid Interest Self-

efficacy Fixed Intell

Low Failers .08 .03 .09 .03 .07 -.02 -.18 Success-

abandon High Failers .01 .03 .12 .24 -.11 .16 .23

Low Failers -.12 -.07 -.12 -.09 -.13 .01 .12 Success-

play-again High

Failers -.09 -.06 -.16 -.29 .04 -.23 -.13

Low Failers .15 .15 .12 .18 .19 .03 .19 Success-

resource High Failers .20 .09 .10 .13 .17 .16 -.23

Low Failers -.17 -.13 .11 .26 .03 -.12 .10 Fail-

abandon High Failers -.04 .05 -.07 .06 .11 -.06 .09

Low Failers -.13 -.03 .06 .02 -.13 -.04 .17 Fail-

play-again High

Failers -.13 -.27 -.20 -.28 -.23 -.18 -.19

Low Failers .25 .14 -.14 -.25 .07 .13 -.22 Fail-

resource High Failers .16 .20 .26* .19 .09 .23 .08

116

Table 6.10. Correlations between motivation measures and learning measures, split by condition and high/low failers.

Control EPB Control EPB Control EPB Control EPB Control EPB Control EPB Control EPB Control EPB

Self-Regulated Learning .35 .01 .40* -.12 .17 -.11 .32 -.04 -.17 .20 -.28 .07 .08 -.06 .30 -.22

Mastery .45** .14 .30 -.19 .17 -.07 .14 -.16 -.11 -.03 -.02 .21 .22 -.01 .05 -.35Performance Approach .06 .17 .13 -.17 .14 -.01 -.10 -.06 .20 .09 -.01 -.05 .07 .11 -.21 .15Performance Avoid -0.40* -.05 .35* -.08 -.12 -.27 -.17 .11 .33 .12 -.13 -.08 .17 -.24 .16 .04

Interest .32 .12 .04 -.29 .37* -.32 -.08 -.16 -.19 .05 -.04 .15 .16 -.07 -.01 -.48**Self-efficacy .41 .13 .38* -.36 .29 .08 .00 -.13 -.29 .05 -.07 .14 .07 -.09 .06 -.33Fixed Intelligence -.24 -.05 -.17 .10 -.22 -.06 -.23 .09 .17 -.01 .40* .02 .22 .03 -.29 .19

Learning Gain

Low Failers High Failers

Pretest Failure Rate

Low Failers High Failers Low Failers High Failers Low Failers High Failers

Science Grade

117

Table 6.11. Correlations between motivation measures and in-game behaviors, split out by condition and high/low failers.

SRL Mastery Perf.

Approach Perf.

Avoid Interest Self-

efficacy Fixed Intell.

Control -.11 -.07 -.08 .02 -.03 -.26 -.13 Low Failers EPB .24 .13 .21 .05 .19 .18 -.22

Control .02 .19 .02 .06 .12 .22 .20 Success-abandon High

Failers EPB .00 -.10 .30 .38 -.27 -.01 .23 Control .05 -.02 -.02 -.10 -.01 .24 .06 Low

Failers EPB -.26 -.12 -.19 -.07 -.27 -.18 .19 Control -.09 -.21 .02 -.04 -.17 -.19 -.16

Success-play-again High

Failers EPB -.12 .06 -.50 -.55* .18 -.21 -.02 Control .25 .34 .37 .27 .16 .14 .30 Low

Failers EPB .10 -.01 -.06 .10 .26 -.01 .08 Control .18 .09 -.10 -.04 .16 -.04 -.05

Success-resource High

Failers EPB .23 .07 .33 .29 .17 .38 -.38 Control -.23 .00 .10 .19 .07 -.06 .16 Low

Failers EPB -.09 -.36 .15 .41 -.03 -.22 .04 Control .16 .21 -.05 .14 .34 .14 -.02

Fail-abandon High

Failers EPB -.20 -.09 -.10 .00 -.04 -.24 .21 Control -.09 -.25 .10 .30 -.13 -.04 .26 Low

Failers EPB -.17 .22 .00 -.37 -.14 .01 .05 Control -.19 -.43 -.12 -.40 -.46* -.33 -.10

Fail-play-again High

Failers EPB -.07 -.14 -.29 -.19 -.06 -.06 -.29 Control .26 .15 -.15 -.35 .02 .08 -.30 Low

Failers EPB .25 .13 -.14 -.04 .15 .19 -.08 Control .01 .19 .17 .23 .08 .16 .13

Fail-resource High

Failers EPB .30 .23 .37 .17 .10 .33 .03

118

Table 6.20. Correlations between persistence behaviors and pre- and post-treatment learning measures, split by high/low failers Science Grade Pretest Failure Learning Gain

Low

Failers High

Failers Low

Failers High

Failers Low

Failers High

Failers Low

Failers High

Failers

Success-abandon -.05 .00 -.10 -.09 .07 .25 -.02 .07

Success-play-again .08 .09 .15 .17 -.10 -.29 .05 -.02

Success-resource -.09 -.19 -.18 -.20 .12 .10 -.09 -.13

Fail-abandon -.12 .06 -.14 -.25 .28 .12 -.06 -.11 Fail-play-again .03 -.12 .37* .23 -.15 -.07 .04 .04

Fail-resource .10 .03 -.16 .07 -.15 -.07 .02 .09

119

Table 6.22. Correlations between pre and post learning measures and responses to failure, split by condition and high and low failers.

Control EPB Control EPB Control EPB Control EPB Control EPB Control EPB Control EPB Control EPB

Success-

abandon -.12 -.01 -.09 .16 -.26 .00 -.14 -.05 .28 -.10 .30 .19 -.05 .00 -.01 -.14

Success-

play-again .15 .03 .22 -.16 .26 .09 .08 .31 -.27 .03 -.28 -.32 .05 .04 .07 .22

Success-

resource -.10 -.07 -.357* -.02 .05 -.32 .19 -.407* -.07 .21 .03 .18 .00 -.14 -.20 -.13

Fail-

abandon -.13 -.14 .20 -.05 -.10 -.20 -.12 -.365* .29 .31 -.09 .33 -.04 -.15 .03 -.22

Fail-play-

again -.16 .17 -.14 -.09 .20 .506** .23 .24 .06 -.388* .03 -.19 .05 .09 .00 .05

Fail-

resource .22 -.03 -.09 .12 -.04 -.30 -.09 .19 -.31 .07 .07 -.20 .01 .05 -.03 .19

Low Failers High Failers Low Failers High FailersLow Failers High Failers Low Failers High Failers

Science Grade Pretest Failure Learning Gain

120

CHAPTER 7: CHOICES OF DISCRETE EVENTS

Overview of findings

This chapter explores discrete choice events and their correlation with learning

outcomes, split by condition, high/low failers, and their interaction. Unfortunately,

there were few correlations between discrete events and learning outcomes that met

our stringent .01 criterion, however there were some interesting findings. About 55%

of students’ choices in the learning environment were made up of game play, while

45% were resource use. The least used resources were reading, solutions, and

explanation. This is unfortunate, since the explanation resource was, arguably, the

most “helpful” of the bunch. Opposite our common sense predictions, resource use

tended to negatively predict learning, while game was a positive predictor. It is

possible that we need more refined measures of resource use (rather than just mere

counts) to find a benefit for resource use.

Time on task was the best predictor of learning, of all in-game measures of

behavior. The more time students spent on task, the more they learned. One

interesting finding is that high-failing Control students made more total “clicks” or

choices in the game compared to the other three groups. The total number of clicks

does not correlate with learning outcome, but it is an interesting difference. Perhaps

the high-failing Control group was engaging in more random clicking behaviors.

Analysis Logic

This analysis examines discrete events undertaken in the game and how these

events relate to pre-existing motivation and pre- and post-treatment learning measures.

The game was designed to contain six discrete learning choices: games, reads,

previews, feedbacks, solutions, and explains. This analysis examines the frequency

with which these choices occur and whether engaging in these actions relates to

learning. It also explores how discrete choice behaviors relate to pre-existing

121

differences of motivational traits and prior learning. Another goal of the analysis is to

examine the types of choices students make when they regulate their own learning.

Since this is a completely exploratory analysis, there are no a priori hypotheses

to guide this investigation. However, given the different learning gains for the high-

failing control group uncovered in the prior chapter, we chose to explore whether

choices and their effect on learning outcomes differ by condition, by high- and low-

failing individuals, and by their interaction.

This analysis will examine each factor separately and in the following order:

1. Full sample

2. Condition

3. High vs. low failers

4. Condition by high/low failers

Each analysis will explore two central questions:

(1) How does the factor of interest affect frequencies of choices? To test this,

simple t-tests and ANOVAs will be carried out to compare means across levels of

each factor.

(2) How does the factor of interest affect the relationship between choice

frequencies and learning outcomes? To test this question, correlations will be used to

gain a broad understanding of the relationship between learning and choices for each

level of the factor.

Given the exploratory nature of these analyses, we will use an alpha criterion

of .01 to designate significant effects. Since a large number of statistical tests are

being done (largely correlations), this alpha correction will reduce the number of

spurious effects that occur by chance.

122

7.1 Full sample

7.1.a Frequencies and proportions of discrete choices

To describe in-game behaviors, Table 7.1 displays the total number of choices

made across two days of game play7. Discrete choices include: playing a game,

reading, previewing, viewing feedback on a game played, viewing solutions for played

games, and reading explanations of those solutions. The “Any Resource” category

lumps together all choices except game play.

The frequency of each choice type was converted to a proportion out of the

total number of choices. Computing proportions essentially controls for the total

number of choices made by each individual. For instance the games proportion is the

total number of games played divided by the total number of choices made, over two

days of game play. On average, students made about 62 discrete choices. 55% of

these choices were constituted by game play, 45% were some type of resource.

Previews and feedbacks made up 39% and 35% of all resource use, respectively.

Reading, checking solutions, and viewing explanations of those solutions made up the

remaining 17% of resource use. Of these, explanations were visited the least

frequently.

Table 7.1. Sums and proportions of choices and failures

Sum Proportion

Games 32.18 (1.01) .55 (.02)

Reads 3.34 (0.42) .06 (.01)

Previews 11.39 (1.13) .16 (.01)

Feedbacks 10.25 (0.69) .16 (.01)

Solutions 3.21 (0.30) .05 (.005)

Explains 1.39 (0.17) .02 (.002)

Any Resource 29.58 (1.76) .45 (.02)

7 All choices were highly correlated across days 1 and 2 of the treatment, so they were collapsed.

123

Other interesting metrics of game behavior include the total number of choices

made (M = 61.77, SE = 2.09) and total time on task8 (M = 49.40, SE = .54).

7.1.a Correlations with learning gains

To get a feel for the relationship between discrete choices and learning,

proportions of each choice were correlated with learning gain scores. Proportions

were chosen for use in all analyses because they tend to be fairly normally distributed,

while the distributions of count data are often positively-skewed.

Table 7.2 in the appendix displays correlations between the proportions of

various choice types and learning gain score. None of the correlations are significant

at the .01 level, however, there are some surprising trends. Learning gain has a small

positive relationship with games played and very slight, non-significant negative

relationships with each type of resource. Summing across all resources (any resource)

reveals a negative correlation between resource use and learning gains. One might

suspect that resource use would be positively correlated with learning, since the intent

of the resources was to help students learn, so this is a puzzling result.

The total number of choices made is not correlated with learning. Total time

on task is one of the strongest predictors of learning gains, indicating that more time

on task was associated with greater learning gains (r = .34, p < .01).

7.1 Summary

These analyses revealed some interesting descriptions of in-game behavior.

First, the majority of students’ choices were made up of game play. Other choices

were comprised of various types of resources; the most often-used were previews and

feedbacks. The “explains”, which provided very specific advice on how to solve

problems and which required a deeper level of processing than many other resources,

were rarely used.

8 Total time on task is the total amount of time students spent across each of the six learning choices in the game. While all students spent the same amount of time at the computer, some students spent more of that time on the homepage (which was not included as one of the six learning activities).

124

Correlations between discrete, in-game choices and learning gains were small

and non-significant, suggesting that discrete choices were not excellent predictors of

learning from the game. Though non-significant, resources tended to negatively

predict learning, while game play was a positive predictor. The best behavioral

predictor of learning was time on task. The more time students spent engaging with

the game, the more they learned.

7.2 Condition effects

7.2.a Frequencies and proportions of discrete choices

This section examines whether in-game behaviors and their relationships with

learning gains differ by condition. Table 7.3 in the Appendix shows the total number

and proportion of discrete choice types split by control and EPB conditions. There

were no significant differences between conditions in either the sum or proportion of

each choice type.

7.2.b Correlations with learning gains

To determine whether the relationship between learning and choice patterns

differs by condition, correlations between each in-game behavior and learning were

computed, as shown in Table 7.4 in the Appendix. The only significant predictor is

time on task, which is a positive predictor of learning in both conditions (rControl = .33,

rEPB = .37, p’s < .01).

7.2 Summary

Analyzing choice patterns separately by condition did not reveal any strong

(nothing met the .01 criterion) differences between groups. Condition did not affect

the way students distributed their choices in the game nor the relationship between

choices and learning outcomes. Time on task is positively related to learning in both

conditions.

125

7.3 High and Low Failers

7.3.a Frequencies and proportions of discrete choices

High and low failers show many interesting differences in choice patterns, as

depicted in Table 7.5. Given that high failers make slightly more total choices (though

not significantly so), proportions should give a more accurate depiction of differences

between groups. For high failers, games make up a smaller proportion of their

choices, while feedbacks and solutions make up a slightly larger proportion. High

failers also spend less time on task. This is consistent with the general pattern that

learning gain correlates (which is correlated with failure) positively with game use and

time on task and negatively with solutions and explains.

Table 7.5. High and Low failers’ mean sums and proportions of choices. Sums Proportions Low Failers High Failers Low Failers High Failers Games 33.07 (1.12) 30.72 (1.49) 0.60(0.02)* 0.49 (0.02) Reads 3.16 (0.64) 3.27 (0.47) 0.05 (0.01) 0.06 (0.01) Previews 11.29 (1.31) 12.41 (1.73) 0.17 (0.02) 0.16 (0.02) Feedbacks 7.49 (0.65)* 13.39 (1.04) 0.12 (0.01)* 0.20 (0.01) Solutions 2.46 (0.35)* 4.19 (0.48) 0.04 (0.004)* 0.06 (0.01) Explains 1.04 (0.19) 1.81 (0.29) 0.02 (0.003) 0.03 (0.004) Any Resource 25.45 (2.17)* 35.07 (2.54) 0.40 (0.02) 0.51 (0.02) Total Choices 58.52 (2.44) 65.78 (3.11) 58.52 (2.44) 65.78 (3.11) Time on Task 52.40 (0.64)* 46.60 (0.72) 52.40 (0.64)* 46.60 (0.72)

7.3.b Correlations between choices and learning gains

Choice patterns of high and low failers were not very predictive of learning

gains, according to the correlations shown in Table 7.6 in the appendix. There is only

one significant, positive correlation between time on task and learning in the high-

failer condition (r = .31, p < .01).

7.3 Summary

High and low failers did have some interesting differences in the way they

approached the game. Low failers played more games and accessed fewer post-game

resources (solutions and feedbacks). They also invested more time into the game

126

overall, while high failers, by inference, must have spent more time on the homepage

(given that all students spent equal time on the computer). Perhaps high failers spent

more time deliberating between different levels of the game or choosing where to

click, or perhaps they were simply less engaged with the game overall. Given that

high failers experienced quite frequent failure (74% of games were lost), it is plausible

that this continued experience of failure would discourage time and effort spent on a

“losing battle.”

7.4 Condition by high/low failers

7.4.a Proportions of discrete choices

To examine differences in how students apportioned their “clicks” in the game,

Table 7.7 in the appendix displays mean proportions for various choice types, split by

high/low failers and condition. For the sake of simplicity, only proportions of choices

are depicted in Table 7. Given differences in the total number of choices across

groups, proportions seemed a more accurate measure for comparing discrete choices.

Two-way, factorial ANOVAs crossing the factors of condition by high/low failers

predicted each choice type or outcome measure listed below. Significant interactions

are marked with an asterisk. The only significant interaction occurred for the total

number of choices; the high failing control group made marginally more total choices

than any other group. One hypothesis is that high-failing control students who did not

have access to an ego-protective buffer, did more “clicking around” to either search

for a game they could actually pass or simply “kill time” until the end of the period.

Anecdotally, the experimenter noticed a set of students who seemed to be aimlessly

clicking on pages without engaging with them; this appeared to be a socially

acceptable way for these kids to opt out without actually quitting. Perhaps these were

the high failing control students, and this is showing up in their higher total number of

choices.

127

7.4.b Correlations with learning gains

Correlations between learning gains and discrete choices and in-game

behaviors do not reveal any striking differences across the four groups. There is only

one significant correlation: time on task is positively correlated with learning gain for

the high failing EPB group. See Table 8 in Appendix for all correlations.

7.4 Summary

A few insights were uncovered by this condition by high/low failers analysis.

The high-failing Control group made more total choices than the other three groups.

One explanation is that these students who were experiencing frequent failure

attempted to opt out of the game by randomly clicking, while high-failing EPB

students did not have the same failure avoidance tendency. Interestingly, time on task

is particularly predictive for the high-failing EPB group, where more time on task

predicts learning. This is the same pattern that was uncovered in the condition

analysis, though it seems to be more exaggerated in the high-failing group.

Discussion

This chapter uncovered some interesting patterns with respect to general

resource use. Students, on the whole, used a small proportion of each resource,

particularly explains, solutions, and reads. Oddly, resource use tended to be

negatively (though not significantly) related to learning gains. Even when pretest

scores and failure rates are partialled out, some of these negative correlations remain.

One interpretation is that the resources are not well-constructed. Another

interpretation is that our measures are not capturing nuanced enough behavioral

sequences. For example, it may be that resource use is only helpful if it lasts for

longer than 1 minute.

Not surprisingly, time on task was positively correlated with learning. No

measures were significant when the data were broken out by condition. Low failers

did show some interesting differences. They spent more time on task, were more

likely to play games and less likely to check a resource, relative to high failers.

128

Finally, the high-failing control group made significantly more total choices.

However, total choices is not correlated with learning gains. So this is not a plausible

explanation for the lower learning gains in the high-failing control group.

129

Appendix to Chapter 7

Table 7.2. Correlations between choice types and learning learning gains

Games .21 Reads -.08 Previews -.09 Feedbacks -.14 Solutions -.11 Explains -.01 Any Resource -.21

* significant at p < .01

Table 7.3. Sums and proportions of in-game choices and failures, split by

condition.

Sum Proportion Choices Control EPB Control EPB

Games 32.57 (1.13) 31.13 (1.52) 0.53 (0.02) 0.56 (0.02) Reads 3.10 (0.57) 3.34 (0.54) 0.04 (0.01) 0.06 (0.01) Previews 13.79 (1.74) 9.92 (1.30) 0.18 (0.02) 0.15 (0.02) Feedbacks 11.99 (1.02) 9.08 (0.83) 0.17 (0.01) 0.15 (0.01) Solutions 3.71 (0.47) 3.00 (0.39) 0.05 (0.01) 0.05 (0.01) Explains 1.67 (0.28) 1.21 (0.22) 0.02 (0.004) 0.02 (0.003) Any Resource 34.25 (2.69) 26.55 (2.06) 0.47 (0.02) 0.44 (0.02) Total Choices 66.82 (3.03) 57.68 (2.54) N/A N/A Time on task 49.90 (0.81) 48.89 (0.72) N/A N/A

* significant at p < .01

Table 7.4. Correlations between proportions of discrete choices and learning gains Control EPB Games .20 .22 Reads .13 -.26 Previews -.01 -.17 Feedbacks -.23 -.02 Solutions -.23 .04 Explains -.12 .14 Any Resource -.20 -.22 Total Choices -.03 .10 Time on Task .33* .37*

130

Table 7.6. Correlations between proportions of discrete choices and learning gains, for low and high failers Low Failers High Failers Game .15 .10 Read -.10 -.03 Preview -.16 -.07 Feedback .02 -.03 Solution -.07 -.01 Explain .07 .02 Any Resource -.15 -.10 Total Choices .00 .12 Time on Task .11 .31*

* significant at p < .01

Table 7.7. Proportions of various activities by high/low failers and condition

Low Failers High Failers Significant

interactions? Choices Control EPB Control EPB

Games .61(.03) .59

(.03) .46

(.02) .52

(.03)

Reads .05(.01) .05(.01) .04

(.01) .08

(.02)

Previews .16

(.02) .18

(.02) .19

(.03) .13

(.03)

Feedbacks .12

(.01) .12

(.01) .22

(.02) .18

(.02)

Solutions .04

(.01) .04

(.01) .07

(.01) .06

(.01)

Explains .02

(.004) .01

(.004) .03

(.01) .02

(.01) Any Resource

.39 (.03)

.41 (.03)

.54 (.03)

.48 (.03)

Total Choices 58.55 (3.69)

58.50 (3.28)

73.82 (4.38)

56.83 (3.94) p = .05*

Time on Task 53.45 (1.04)

51.44 (.77)

46.89 (1.00)

46.27 (1.07)

* significant at p < .05

131

Table 7.8. Correlations with learning gain score by high/low failers and condition Low Failers High Failers Control EPB Control EPB Games .10 .20 -.24 .21 Reads -.04 -.20 .34 -.27 Previews -.13 -.17 .23 -.24 Feedbacks .18 -.13 -.11 .13 Solutions -.17 .05 -.14 .11 Explains -.07 .26 .05 .07 Any Resource -.10 -.20 .24 -.21 Total Choices .09 -.14 .17 .26 Time on task .06 .13 .21 .46*

* significant at p < .01

132

CHAPTER 8: DISCUSSION

This study explored the importance of persistence in the face of failure with

respect to learning. Seventh grade students were immersed in a computer-based

genetics game that contained a plethora of learning choices. Students could choose

what game topic to learn, what resource(s) would help them learn it, and when to

move on to a different topic. Given the opportunity to regulate their own learning,

persistence was predicted to be a crucial factor in moderating students’ learning

outcomes. On the assumption that failure can signal an opportunity to learn, we

predicted that persistence after failure would be particularly instrumental in facilitating

learning.

Prior work by the author suggested that making a combination of internal and

external attributions for failure might help students persist. We designed an

intervention to invite students to attribute failure to both internal and external causes.

We call this an ego-protective buffer (EPB), because the external attributions are

meant to protect students’ egos from the negative psychological ramifications of

failure. However, we expected students to make some internal attributions as well,

which should help them take responsibility for remedying the failure situation. The

mixed attributions encouraged by an EPB are meant to increase persistence after

failure, which should, ultimately, enhance learning from the game. We also

speculated that an EPB might encourage more productive persistence, whereby

students would choose to engage in better learning strategies.

Review of empirical findings

There were two strands of analyses. One strand focused on the effects of the

EPB intervention, asking the following central research questions: (1) Does an EPB

promote persistence after failure? (2) Does persistence after failure influence learning?

(3) What is productive persistence after failure? (4) Does an ego-protective buffer

promote productive persistence?

133

A second strand of analyses explored the possibility of using students’ learning

choices to measure self-regulated learning and motivation. Here we focused on

behaviors related to persistence and other discrete learning choices in the game (i.e.

which learning resources students chose). In-game behaviors may assess constructs

that are not captured by traditional motivation or self-regulated learning measures but

which are, nonetheless, consequential for learning. Analyses focused on the following

exploratory research questions: (1) Do pre-existing individual differences predict

persistence behaviors? (2) Do persistence behaviors explain some variance in learning

that cannot be explained by pre-existing individual differences? (3) More generally,

which learning choices (beyond persistence) predict learning outcomes?

Below, we summarize the current findings around these research questions,

discuss theoretical and practical implications of the work, and propose future

directions for the research.

The effect of the ego-protective buffer on persistence behaviors and learning outcomes

A four-pronged analysis investigated the central hypotheses. Visiting the

hypotheses by partitioning the data in different ways helps to separate real effects

from “fluke” effects.

The first analysis explored treatment effects on learning, persistence behaviors,

and the relationship between persistence behaviors and learning. The results indicated

that a lack of persistence after failure (represented by the fail-abandon behavior) was

associated with lower learning gains for all subjects. Moreover, the association was

not just between persistence and learning, but persistence after failure specifically.

Persistence after success had no relation to learning gains. We interpret this to mean

that persistence after failure is critical for learning.

With respect to treatment, students in the Control condition, who did not have

access to an ego-protective buffer, were more likely to quit after failure and persist on

a level of the game where they experienced success. One interpretation of this result

is that without an ego-protective buffer, students tended to flee from past failures and

avoid future failures. In other words, they displayed less persistence after failure and

134

they made choices that reduced their risk of future failure. In contrast, students

provided with an EPB tended to quit at the same rates after both successes and

failures. EPB students behaved as if they were “buffered” from the negative

ramifications of failure. However, the higher rate of persistence after failure in the

EPB condition did not translate into overall learning differences by condition.

Students in the EPB and Control conditions made similar learning gains from pretest

to posttest. Given that the persistence after failure was a correlate of learning gains

across the full sample, and the EPB condition showed more persistence, this raised the

question of why there was not an overall learning advantage for the EPB condition.

Given that the ego-protective buffer was designed to protect students from the

negative psychological ramifications of failure, we speculated that the intervention

might have been most effective under conditions of high failure. So we examined

learning and behavioral outcomes for high- and low-failing students across both

conditions. High-failing students in the Control condition learned far less than their

low-failing counterparts. In contrast, high- and low-failing students in the EPB

condition learned the same amount. The ego-protective buffer did influence learning,

but only for those students who experienced a great deal of failure.

To explore the mechanism behind this effect, we examined persistence after

failure and success for the high- and low-failing students in each condition. Once

again, Control students tended to flee from failure but persist after success. This

pattern was exaggerated in the high-failing Control students. While this was only a

descriptive pattern, it offers some explanation for the learning effects. High-failing

Control students were less likely to persist after failure than their low-failing

counterparts, and so they learned less.

Given that the effect of the EPB was particularly strong for students who

experienced more failure, we conducted similar analyses on a within-subjects level, to

see whether all students would show similar effects when they encountered a high

failure situation. The within-subject results were parallel. The pattern of persistence

behaviors that occurred across conditions (i.e. Control students are more likely to flee

failure and persist after success) was particularly robust in high-failing situations.

135

However, the learning effects were weaker and non-significant. In other words, the

ego-protective buffer was more effective at increasing learning in high-failing students

than in high-failing situations.

To further isolate the significance of the response to failure, we examined

whether there were similar results when the data were partitioned by hard and easy

game levels. It may have been the case that the key variable was not failure per se,

but rather the challenge of the problem. For example, students with low prior

knowledge for a particular game level might have abandoned the level after

discovering how hard the game was. In this case, the effect would be based on the

experience of challenge, rather than the experience of failure. To find out, we did one

last analysis to compare students’ persistence and learning gains in situations of high

and low challenge, across conditions. High challenge situations were those game

levels for which students had done poorly on the pretest. We found no learning

differences across high and low challenges situations. However, we did find the same

pattern of persistence behaviors that occurred in previous analyses. Compared to EPB

students, Control students showed less persistence after failure and higher rates of

risk-averse choices in high challenge situations. However, these effects were

generally more modest than the effects gained by partitioning the data according to

failure rates.

Generalizing across these four analyses, which occurred at both person and

situation levels, and which examined learning across the entire game and learning on

specific game levels, we came to the following conclusions. The most robust finding,

which occurred either descriptively or significantly in every analysis, was the effect of

the EPB on behaviors in response to and in anticipation of failure. An ego-protective

buffer encouraged persistence following failure and risk-taking (i.e. EPB students are

more likely to try a different game level after success). We also found that the ego-

protective buffer had an impact on learning, particularly for high-failing students,

though the trends were the same for high-failing situations. Students who were not

provided an ego-protective buffer and who failed often learned far less than low-

failing students who were given the same buffer. An ego-protective buffer may not be

136

necessary for students who experience a small amount of failure. Perhaps low-failers

are more resilient to failure and do not need an external source of motivational

support.

Another interesting finding is that prior achievement and failure in the game

are marginally better predictors of learning in the Control condition than in the EPB

condition. This suggests that learning in the EPB group is divorced from prior

achievement; in a sense, the EPB manipulation has “leveled the playing field” for high

and low-achieving students. Also, learning in the EPB condition is less sensitive to

failure in the game; EPB students were more resistant to the effects of failure on

learning.

It is important to note that the effects of the ego-protective buffer with respect

to learning are relatively small compared to the effects of prior knowledge.

Persistence after failure accounts for about 4% of the variance in learning gain scores.

Plus, the effect of the EPB on the high-failing students versus the low-failing students

also explains 4% unique variance in learning gains. At the same time, 4% is worthy of

attention, especially when it comes to models of instruction that can reach millions of

children.

An exploration of productive persistence and treatment effects

When students do persist after failure, what kinds of behaviors do they engage

in, and which ones are productive for learning? The analyses in Chapter 5 were

designed to answer these questions, while investigating how productive persistence

might differ across conditions and students who experience more or less failure. (We

focused on high and low-failing students, rather than situations, because this was the

locale of the strongest effects in the prior analyses.) Persistence behaviors were broken

into two categories: playing the game again or checking a relevant resource. While we

had predicted that checking a resource would be productive, neither of these categories

of persistence was a significant predictor of learning gains. Only persistence after

failure itself, as indicated by not abandoning the level after failure, had a significant

impact on learning.

137

We also discovered that what students did when they persisted did not differ

between conditions or between the high- and low-failing students across conditions.

Nor did it have a differential effect on persistence type behaviors. So, while the EPB

does encourage high-failing students to persist after failure, it does not encourage any

particular type of persistence.

Additional analyses were conducted to examine the behaviors preceding

success and failure, however these will not be discussed, because the findings do not

shed light on the main research questions.

The effect of pre-existing individual differences on persistence and learning

A second line of analysis was focused on the more practical concern of

whether in-game behaviors could be used as measures of motivation or self-regulated

learning. One idea is that micro-behavioral measures may capture some aspects of

situational motivation that cannot be predicted by pre-existing differences in more

global, trait-focused scales of motivation. For instance, persistence after failure,

which was captured by the reciprocal behavior of leaving a game level after failing,

was correlated with learning gains. However, persistence after failure was not

predicted by any of the seven motivational survey measures that were given before the

treatment. The motivational scales were not capturing this persistence behavior, even

though many of these motivational constructs are associated with persistence after

failure. For example, intrinsic motivation, self-efficacy, and mastery motivation

theories claim that students high in each construct should be more likely to persist

after failure. The fact that these measures do not predict the fail-abandon behavior

shows that measures of actual behaviors can capture useful aspects of situational

motivation, like persistence after failure. Moreover, persistence in the face of failure

has consequences for learning. So in addition to measuring situational motivation, the

fail-abandon behavior captures behaviors that are indicative of learning outcomes.

138

Discrete choices in the game as predictors of behaviors and learning

The next analysis examined discrete choices in the game, in an effort to (1)

examine students’ self-regulated learning in an open-ended environment and (2)

explore whether students’ learning outcomes could be predicted by their behaviors in

the game. The proportion of in-game choices dedicated to each type of learning

activity is not significantly predictive of learning gains. However, the general trend is

that high resource use predicts low learning gains, even when failure and pretest

scores are partialled out. In contrast, high frequency of game play predicts high

learning gains. Also, time on task is one of the strongest predictors of learning gains.

Given that the resources were designed to teach the content covered in the

games, these results are fairly surprising. Perhaps other measures of resource use

would show a different relationship with learning. For instance, if instead of counting

resource “clicks,” we counted “meaningful clicks,” where students went to a resource

and spent a reasonable amount of time there, then perhaps resource use would

positively predict learning. Of course, it is possible that the resources simply did not

help students learn the complex genetics content and that, instead, students learned

more effectively through practice and trial and error (i.e. game play).

The analysis of discrete choices did give us some idea of how students regulate

their own learning in a choice-filled environment. In general, students spent

proportionately very small amounts of time on the resources that we would expect to

improve learning. On average, 6% of all choices were reading choices, and only 2%

of choices were deep-level explanations of the correct answers to a game. Similar to

findings from past research on help-seeking, students rarely sought help (in the form

of resources), and they were less likely to seek help that involved deep processing

(Aleven et al., 2003). It is somewhat surprising that students rarely chose to receive

an ‘explanation,’ which provided context-sensitive help for specific aspects of a game.

A priori, one would assume that these are the most pointed and efficient resources for

learning how to win a game. Thus, whatever the case about resource use, these

students did not seem particularly high on the metacognition needed to decide on the

139

best resources for their learning, which may help explain why persistence after failure,

rather than persistence with the use of a resource, predicted learning.

Our interpretation of the findings

With respect to the research questions around ego-protective buffers and their

impact on learning and behavior, many of the original hypotheses were either

supported or qualified. As predicted, the ego-protective buffer did encourage

persistence after failure. However, we did not predict that it would be exaggerated

under conditions of high failure nor that it would encourage risky behaviors.

We had original predicted that the ego-protective buffer group would learn

more than the Control group. Provision of an ego-protective buffer did lead to greater

learning gains, but only amongst high-failing students. The one hypothesis that was

clearly refuted is that an ego-protective does not encourage more productive

persistence, only more persistence overall.

What is the mechanism behind the ego-protective buffer? The idea for the

ego-protective buffer was originally borne from our belief that a combination of

internal and external attributions would enhance persistence after failure. The results

are consistent with the attribution-related theory that predicted them, but it is

impossible to link the results directly to attributions as the mechanism. The interview

data, which we have not presented as part of the thesis, focused on attributions for

winning or losing a game and may provide more direct evidence. In future research, it

would be wise to embed attribution-related, Likert scale survey measures within the

game, to get reliable, situation-focused attributions in response to success and failure

in the game, without having to code verbal data.

Alternative interpretations

Beyond our attribution-based theory, there are certainly alternative

interpretations of the results and the mechanisms at work behind them. For instance,

one slightly alternative explanation is that students become high-failers precisely

because they abandon. In this scenario, it is not students’ response to high failure that

140

drives the learning effects. Rather, it is their, perhaps pre-existing tendency to not

persist that causes them to experience high failure. In general, this is one of the

problems of correlational data; it is impossible to determine which is the cause and

which is the effect. In any case, this alternative explanation is not completely at odds

with the attribution theory explanation, it just changes the story a bit. Instead of high

failure being a cause of fail-abandon, fail-abandon is a cause of high failure.

An alternative interpretation is that students’ behavior and learning in the EPB

condition is better explained by the nature of gambling than by attribution theory. For

instance, the element of chance in the game may lead students to view it as a gambling

type activity. In the EPB condition, where students believe it is possible to win with

only one correct answer in a game, students may be more willing to try games they

have previously failed because there is a chance they could win. Likewise, they may

be more willing to risk a different level of the game, even when they have found a

level where they can succeed, because their odds of winning a new game are based on

chance. However, it is unclear why high-failing students would buy more into this

gambling idea in the EPB condition. So this explanation does not hold up under closer

scrutiny.

The most plausible alternative explanation is that the Control condition has

been primed to adopt a performance-avoid goal. By providing them with the 75% rule

for winning, we have essentially given the Control group a concrete standard to shoot

for – a goal for their performance. In addition, the game provides rewards (points)

when students meet the 75% standard and punishes (takes away a “life”) when

students do not. This environment contains strong negative consequences for losing a

game, which may have primed students in the Control condition to adopt a

performance-avoid orientation. (In contrast, the EPB group did not have a fixed

performance goal, because of the chance element.) Students who adopt performance

avoid goals often persist less after failure and choose less challenging tasks. However,

these behavioral effects usually appear in challenging situations, such as high failure.

This could explain why the high-failing Control students persisted less after failure

and ultimately, learned less. In this scenario, the EPB condition does not have a buffer

141

to protect them from failure, but rather, they do not have a clear performance goal.

However, if this were true, we would expect the performance-avoid survey measures

to correlate positively with fail-abandon behaviors, particularly in the EPB condition,

but this was not the case. So this explanation does not adequately fit the findings.

This chapter reviewed the main findings, our basic interpretation of them, and

presented some alternative hypotheses. The next chapter explores the broader

implications of this work and future directions.

142

CHAPTER 9: RELFECTIONS, IMPLICATIONS, AND FUTURE

DIRECTIONS

This research begins to explore motivational supports for learning from failure

situations. Failure often presents a valuable learning opportunity, and in fact, many

instructional models have capitalized upon it. However, in order to learn from their

mistakes, students may need motivational scaffolds to protect them from the negative

psychological ramifications of failure.

This work explored the effectiveness of a motivation-based intervention called

an ego-protective buffer, that was designed to enhance persistence after failure. An

ego-protective buffer shields one’s sense of competence from the negative

ramifications of failure. The particular kind of EPB tested in this study was designed

to work by eliciting a combination of internal and external attributions for failure.

External attributions protect one’s sense of competence by averting the blame for

failure away from the self. This should discourage one from quitting the task. At the

same time, this ego-protective buffer invites some internal attributions, which

encourages students to take some responsibility for remedying the failure situation.

Based on this theory, we embedded an EPB into the rule structure of a game-

like learning environment. In the EPB condition, students were told that winning in

the game was a probabilistic outcome, dependent on a combination of chance and skill

on the part of the students. In the Control condition, students were told that winning

in the game was a deterministic outcome, dependent on students’ skill only. The EPB

did have an effect on learning, but only amongst high-failing students. High-failing

EPB students learned just as much as their low-failing counterparts. This was not so

in the Control condition, where high-failing students learned far less than their low-

failing counterparts. So the high-failing EPB group was behaving as if they were

“buffered” from the effects of failure. We also found evidence of a possible

mechanism behind this learning effect, though the exact connection between behaviors

and learning outcomes is not clear. In the high-failing EPB condition, students were

143

equally likely to persist after success and failure, while in the Control condition,

students were far more likely to persist after success. This was a descriptive

difference that occurred more strongly within subjects in high failure situations.

Finally, persistence after failure was associated with learning across the full sample of

subjects, suggesting that regardless of condition or failure rate, students who persisted

more after failure also learned more.

This study adds to the growing body of evidence in support of the persistence-

provoking and learning-boosting effects of ego-protective buffers that invite internal-

external attributions. Three sets of studies – the Teachable Agent (TA) studies, the

expert study, and the dissertation study – all set in drastically different settings, with

different types of failure, and different age groups – have shown the same result: an

ego-protective buffer is associated with greater persistence after failure. Since these

three studies all took place in drastically different contexts, they attest to the generality

of this type of internal-external EPB. In the TA studies, students were led to believe

they were teaching a digital pupil, which invited them to place some of the blame for

failure on their tutees. In the expert study, experts failing on a task in their domain of

expertise spontaneously attributed failure both to themselves and to the environment.

In the dissertation study, students believed that winning in the game was due to a

combination of skill and chance. Each of these studies contains a different

instantiation of the EPB, and yet the results are fairly consistent. Taken as a whole,

these studies provide compelling evidence that the EPB is a useful motivational

support that does enhance persistence.

The dissertation study attempted to isolate the effect of the EPB and test its

effectiveness in a new context and with a new implementation (chance-based).

Unfortunately, the attributions students made towards failure in the game were not

analyzed, so we cannot claim that the designed ego-protective buffer successfully

elicited mixed attributions for failure. However, we did find that the intervention

evoked persistence after failure and encouraged more risk-taking, particularly under

conditions of high failure. And both the current study and the pilot study showed that

persistence after failure is critical for learning.

144

The next section discusses some reflections on the game context in which the

study is set and the use of chance as an external cause of a failure outcome. Then, the

work’s many implications for theory-building, the design of instruction, and the

design of behavioral learning measures will be discussed. Finally, the remainder of

this chapter will explore potential future directions.

Reflections

The game context

Because the study took place in the context of a game rather than a more

realistic school learning activity, it must be acknowledged that the effects might be

unique to the game context. For instance, if the same study were done in the context

of taking a test, where students often attribute the outcome to ability, students might

be far less likely to make external attributions to chance, and hence, the effect of the

EPB on persistence could be lessened. In general, caution must be used when

interpreting the results, since these findings may not generalize beyond the context of

this particular genetics game.

However, the game context could also be dampening the effects. An ego-

protective buffer is probably most necessary when students are highly ego-involved in

the task. That is, the ego needs to stand a chance of being boosted or hurt by the

outcome for students to become ego-invested. A game is low stakes; since it is not

impacting students’ grades, loss of the game could easily be shrugged off as “no big

deal.” Whereas, had the failure been more consequential, where it affected students’

grades or somehow had the potential of affecting one’s ego in a more demonstrative

way, the effect of the ego-protective buffer might have been even more pronounced.

Another alternative would be to make students more ego-invested in the game

itself by making the game more like real life. Currencies instead of points, real-world

tasks instead of alien puzzles, and interaction with other students in the game might

have served to make the game more realistic. If students had cared more about how

they performed in the game, they might have had a greater need for an EPB.

145

The game context could be an interesting place to house an assessment. The

data described in this dissertation show that behaviors engaged in during game play do

predict learning outcomes. Failure rate in the game itself is also an excellent predictor

of learning. These data suggest that performance and behaviors in the game can

forecast performance on a paper-and-pencil test. However, it is possible that if we

told students they were being assessed in the game that they would behave or perform

differently. This would be an interesting study to conduct. In one condition, students

could be told that they are playing a game, while in another condition, they are told

they are taking a test. Behaviors would undoubtedly change but the same types of

behaviors may still be predictive of learning, in both contexts. If behaviors in the

game turned out to be fairly good predictors of student performance on more standard

assessments, then educational games could be a relatively stress-free way to assess

students’ knowledge.

Attributions to chance

This study’s particular instantiation of the EPB used chance as an external

cause for the outcome. However, many other types of external causes could be used.

For instance, the Teachable Agent studies used another person (the agent) as the

external cause. A common external attribution for performance in school contexts is

the difficulty of the task. Chance has particular properties that make it different from

other kinds of external attributions. Chance is both uncontrollable and changeable.

While almost all external attributions are uncontrollable by definition, one’s

luck can change. The possibility of change could provide some hope that things will

get better, if students try again. This could explain why the EPB affected persistence

in this study but not productive persistence. It makes sense that the chance outcome

would not encourage more productive persistence, since it does not point to effort or

strategy as a source for the failure outcome. Whereas, if students believed that the

outcome was malleable and might change by chance, they might have been driven to

persist by simply trying again, without taking up more productive learning actions.

146

Are attributions to chance healthy for students? Most attribution researchers

argue that effort is the healthiest attribution to make because it makes the student

responsible for remedying the outcome and encourages persistence at the task.

However, there are times when the outcome is partially due to chance, and in those

cases, it might be better for students to understand that a failure is not totally their

fault. For instance, when graduate students submit a paper for publication, the

reception of their paper partially depends on the particular reviewers who are chosen

to assess the paper’s readiness for publication. In that sense, there is an element of

luck involved in whether the paper is rejected or accepted. Other outcomes, like

scientific experiments are also influenced by random variation. In these examples,

making a failure attribution to effort alone could bruise a student’s ego, so partial

attributions to chance are warranted.

Attributing failure to chance alone (or any external cause) would be an

unhealthy attribution to make in a school context, since it gives no responsibility to the

student. For instance, learned helpless individuals often perceive that external,

uncontrollable factors (like chance) have a strong impact on their performance, and

they display extremely negative learning behaviors (Diener & Dweck, 1978). Thus, it

is important that students make only partial attributions to chance. Making some

attributions to internal causes (like effort or knowledge) is critical for persistence. The

internal-external EPB thrives on this balance.

Implications

Implications for theory-building

This dissertation study had the potential to contribute to three pieces of theory

related to the ego-protective buffer concept: the mechanisms behind the effect, the

conditions under which the EPB is effective, and a more precise understanding of the

behavioral outcomes. With respect to mechanisms, we originally proposed that the

EPB used in this study would elicit a combination of internal and external attributions.

By simultaneously shielding students from negative self-blame and encouraging them

to take responsibility for remedying the failure, mixed attributions would enhance

147

persistence after failure. This is a plausible story, but without attributional data, it is

impossible to provide support for this hypothesis. However, there are two pieces of

indirect evidence that make the attribution theory seem probable. First, two other

studies which also contained situations where the ownership of outcomes was

ambiguous – the TA study and the expert study -- demonstrated that a combination of

internal and external attributions coincided with greater persistence after failure.

Second, all the predictions that were derived from the attribution-based theory of the

EPB were either directly supported or mildly qualified by the dissertation study’s

findings. Looking at the evidence as a whole, it is surely feasible that the type of EPB

tested here supports persistence by encouraging blended attributions for failure.

If we assume that the EPB did work as intended, then this dissertation study

further refines the conditions under which the EPB works. We originally predicted

that all students would benefit from the support of an EPB, but in this study, the

internal-external EPB was most effective under conditions of high failure. High

failing students showed the strongest learning effects from the EPB, while students in

high failure situations showed stronger persistence effects from the EPB. These

effects need to be teased apart. Do high-failing students always persist less in failure

situations? Are they more prone to engage in fail-abandon behaviors and thus, stand

to benefit more from the support of an EPB? Or do high-failing students have some

other trait, like low motivation, that is causing them to benefit more from the EPB?

To explore this alternative interpretation in the current data set, we could test this by

comparing students with high and low motivation (based on a composite of pre-survey

measures) and determine whether they show the same pattern of persistence and

learning as high and low-failing students. To do a more controlled study that would

test the effects of high-failing students against high-failure situations, we could split

students into groups of high and low-failing students based on one session of game

play (with no EPB). Each of those groups of students could then be exposed to high

or low failure situations (25% vs. 75% failure) in the game, with and without access to

an EPB. The resultant patterns of persistence behaviors and learning outcomes in each

group might help tease apart the effect of failure on the effectiveness of the EPB.

148

Another interesting question raised by the dissertation study is whether certain

external attribution types are more effective at promoting persistence after failure. In

this dissertation study, the internal-external EPB provided to students was designed to

invite partial external attributions to chance. In the TA study, students were invited to

make external attributions to another person (the Teachable Agent character). How

does blaming failure on chance differ from blaming another person? Without the

attributional data, we can only speculate.

However, children should have an easier time reasoning about less abstract

external causes, such as the actions of another person, while probabilistic or chance-

influenced outcomes are more difficult to conceptualize, even for adults (Nisbett,

1987). This might explain why the between-condition learning effects were stronger

in the TA study than in the dissertation study. Perhaps an EPB that provides more

concrete external causes for outcomes can support all individuals and not just the high-

failing ones. A simple experiment could test this by contrasting an abstract and a

concrete cause of failure (e.g. the weather vs. chaos theory) embedded in an EPB.

This work also further refines the behavioral outcomes of an EPB. Though we

had originally predicted the EPB would enhance persistence after failure, this was only

one of the positive impacts it had on students’ behavior in this study. The EPB also

discouraged risk-avoidance behaviors by moving students away from “safe” choices

that would prove their competencies rather than grow them, particularly under

conditions of high failure. This may be unique to EPBs that use chance as an unstable

external attribution. If there is always a chance that students can do well, they might

be more willing to take that chance. Whereas, if the external attribution were to the

difficulty of the subject, which is a stable, external cause, the risk-taking effect might

disappear. This could be tested in a study with two comparable EPBs, which

encouraged external attributions to either stable or unstable causes.

A related possibility is that the internal-external EPB, which was designed to

affect students’ perception of and response to failure, could also be altering students’

perception and response to success. Are combined internal and external attributions to

success a good thing? Some researchers argue that attributing success to the self is a

149

healthy behavior that builds self-efficacy and a sense of competence (Schunk, Pintrich,

& Meece, 2008). Does attributing part of that success to an external cause mitigate

this effect? Or are students so unlikely to make attributions for success that the EPB

would have no impact on self-efficacy? This is an empirical question that remains to

be explored. In this study, post-survey data showed that EPB and Control students

had similar levels of self-efficacy for the game, suggesting that combined attributions

to success did not impact students’ sense of competence at the task. Future analyses

of the interview data could shed light on the types of attributions students make for

success.

In sum, this study contributed to our understanding of the conditions under

which this type of internal-external EPB is effective, it found new behavioral

outcomes of this EPB, and it raised the possibility of several new research directions.

More empirical work and analyses of the current data will begin to scratch the surface

of these theoretical questions.

Implications for instruction

Three studies have now shown that situations which contain an internal-

external EPB are in some way associated with greater persistence after failure. And

two studies have demonstrated a clear link between persistence after failure and

learning. This evidence suggests that the internal-external EPB has potential as an

instructional support that could enhance persistence and learning, particularly under

conditions of high failure.

What are possible instantiations of an EPB? There are two ways to provide an

EPB. The first is to build an EPB into the learning environment. Examples already

discussed include the Teachable Agent online learning environment, where students

can partially blame failure on the agent they are programming, and the genetics game

in the dissertation study, where students believed that outcomes were partially due to

chance. Other common educational practices have built-in internal-external EPBs,

like peer tutoring, collaboration, and educational games. In each of these instructional

situations, responsibility for failure can be distributed across the self and an external

150

cause. This might help explain why these instructional methods are believed to be

effective for motivation and learning.

What other ways could internal-external EPBs be built into the structure of a

learning activity? Computer-based learning environments might trigger EPB-like

attributions for failure, when a high degree of failure is detected. For instance, in a

math-based learning environment, if a student is failing more than 50% of the

problems, this could prompt an on-screen character to make failure attributions to a

combination of internal causes (student effort or strategy) and external causes (like the

difficulty of the task).

In addition to environmentally-based EPBs, students might be able learn to

self-generate an ego-protective buffer under conditions of high failure. In the expert

study, the experts made a combination of internal and external attributions for failure

during the task in their domain. But during the task outside their domain, they made

largely internal attributions. Other research has shown that experts have an excellent

understanding of how external conditions affect their performance (Ericsson, 2002).

For instance, expert tennis players are constantly aware of wind conditions, which

they practice adapting to. One possibility is that experts are able to generate external

attributions for failure because they have developed the ability to perceive the external

conditions that affect their performance in their domain of expertise. In addition to

noticing external causes, the experts in this study also worked around or changed the

external conditions that were hindering their performance (e.g. by eliminating time

constraints, consulting resources, etc.) in order to succeed. This might be a worthy

goal for students to reach.

One possible intervention that might help students notice external causes is to

ask students to reflect on a negative outcome. Students could list all the possible

reasons for the negative outcome, both internal and external. It may also be

productive to have students brainstorm and discuss ways they could adapt to or

prepare for external causes for negative outcomes. The focus of the intervention

would be to help students to (1) learn to recognize when and how external conditions

are affecting their performance and (2) learn how to adapt to these external

151

circumstances. Of course, this intervention (and variants of it that do/do not include

factors 1 and 2) could be tested in the classroom.

These interventions are speculative and need empirical evidence before they

can be unleashed on the classroom. In particular, clear evidence of the attributional

mechanism behind the internal-external EPB is important for designing instruction

that buffers the effects of failure or teaches students to create their own buffers.

Implications for measurement

This study, along with the pilot study that preceded it, provides an existence

proof for the possibilities of behavioral-based measures of learning process and

motivational behaviors. The “fail-abandon” behavior captured in the dissertation study

is a measure of persistence after failure, which is often a behavioral outcome of

motivation. While many different motivational constructs should promote persistence

after failure, in the dissertation study, motivational constructs did not predict

persistence. Behaviors like fail-abandon provide information about motivated

behaviors that (1) are not necessarily captured by survey measures of motivation, (2)

may provide more situational measures of motivation that are more predictive of

learning than global, trait-like measures of motivation, and (3) can be changed by

interventions.

The current research points to the value of capturing motivational behaviors

(like persistence or risk-taking) which occur during the learning activity. Even though

we may not know what motivational construct caused these outcomes, they may still

be worth measuring if they help us predict learning. Moreover, functional measures of

learning behaviors also describe the process of learning. Process data often points out

possible mechanisms or raises new questions about the theory behind the research.

Moreover, having good measures of learning behaviors, particularly ones that

are changeable via interventions, could be very useful in the development of adaptable

learning environments. Computer-based data mining techniques are becoming

increasingly more sophisticated, and it is now possible to collect and process students’

behavioral “clickstream” in real time. For instance, an adaptable learning environment

152

could detect when a student is failing and abandoning too frequently and intervene

with an EPB in the middle of the learning activity. Adaptable learning environments

have enormous potential to impact learning, but to intervene in refined ways, we must

be able to identify and measure healthy and unhealthy learning behaviors. This

dissertation research takes us one step closer in the direction of creating measurements

of critical behaviors that relate to learning and motivation.

Future Directions

This body of work raises a number of questions that suggest three different

lines of future work. The first line of research could shore up several theoretical

claims about the mechanisms behind the internal-external EPB and the importance of

persistence after failure for learning. A second line of work could follow up on the

possibilities of creating computer-based measures of learning and motivational

behaviors. Finally, a third direction would explore the idea of promoting productive

persistence after failure.

Confirm theoretical claims

Three pieces of the theoretical story behind the EPB need better evidence.

First, we need to know whether an internal-external EPB provided to a student during

learning actually does elicit a combination of internal and external attributions.

Second, we need more definitive evidence that a combination of internal and external

attributions can spur persistence. Finally, we need to test the effect of fail-abandon on

learning more directly.

To get at the first link in the chain of reasoning about the internal-external

EPB, there are several possible approaches. One is to code and analyze the interview

data from the current study, to determine whether students given access to an ego-

protective buffer make a combination of internal and external attributions, as

predicted. A second approach is to run another study, this time embedding survey

measures of attributions in the task. For instance, if we were to run the same study

again, survey items could occasionally appear on-screen following success and failure

153

in the game. Survey items add precision to the measure of attributions because they

can get at more specific attributions like fixed intelligence or long-term effort. These

may not be easily identified in verbal data, when kids are wont to make ambiguous

attributions like “I don’t get this” or “I didn’t read the right thing”.

To gather stronger evidence for the claim that combined internal-external

attributions scaffold persistence, a simple study would make the attributions directly

for the students. For instance, students could use the same genetics game, only each

time they won or lost a game, a message would appear on-screen making either an

internal (“Try harder next time.”), external (“This task was really hard.”), or a

combined internal-external attribution (“This task was really hard. Try harder next

time). This could also be implemented in a lab-like setting where an experimenter

verbally states the attribution while students work on math problems, for instance. In

any case, we would expect the combined attribution condition to show the greatest

persistence after failure.

In the dissertation study, we established a correlational link between fail-

abandon behaviors and learning. Any number of other person-level variables that we

did not measure in the study could be co-occurring with fail-abandon, which casts

doubt on fail-abandon as the cause of learning gains. To more carefully establish this

link, we could do a simple study where one group is forced to abandon after every

failure, another group is forced to play again after every failure, and yet another group

is forced to check a resource after every failure. If the fail-play-again and fail-

resource groups make greater learning gains than the fail-abandon group, then we

would have more definitive evidence of fail-abandon’s effect on learning. Moreover,

it would be interesting to see how the fail-resource group fares, given that we expected

them to make the greatest learning gains, but in the current study the fail-resource

behavior does not correlate with learning, and more generally, resource use tends to be

a negative predictor of learning.

These three experiments would provide strict tests of the claims behind the

mechanisms by which the internal-external EPB works and the link between failure

and learning.

154

Measurement of learning and motivational behaviors

One of the limitations of the dissertation study is the lack of constraints in the

design of the environment and behavioral measures. In our zeal to provide students

with the utmost choice and control in their learning, we created a noisy data collection

environment with many uninterpretable sequences of learning choices. To get cleaner

measures of specific types of learning and motivational behaviors, it would be better

to clearly define the behaviors we want to measure and the conditions under which

they are likely to be elicited, a priori. For instance, if we were interested in capturing

functional measures of motivation, we might first create a list of broad behavioral

constructs like persistence, risk-taking, or risk-avoidance, and then operationalize

them as specific behaviors in the context of the learning task and environment.

Similarly, the task should be designed to elicit these behaviors, and even better, we

should have strong hypotheses about how exactly these behaviors will affect learning.

Starting with a strong theoretical model, well-defined hypotheses, clear

operationalizations of behavioral constructs, and an environment designed to measure

them would, hopefully, increase our chances of building effective, computer-based

measures of meaningful process data. Having clear hypotheses and measures would

guide us in finding meaningful sequences of behaviors that are likely to correlate with

learning or motivation.

A second limitation of the dissertation study is that the measures of behavior

are very simplistic. Simple two-event sequences provide a very limited snapshot of

the learning behaviors students engage in. For instance, when a student leaves a level

after failing, this is classified as a fail-abandon behavior, but that student may come

back much later in the sequence. Developing and applying more sophisticated data

mining procedures to the current data set would allow us to capture a whole host of

more complex learning behaviors (like eventually revisiting a game level) that are

more indicative of students’ strategies and self-regulated learning.

Another interesting avenue to pursue is to classify students (rather than their

behaviors) into types. For instance, perusal of the data revealed broad three categories

of game starters. There are explorers, who preview several levels before settling down

155

and playing one. There are researchers, who read and view other resources before

attempting to play a game. And, there are gunners, who jump right into playing a

game, without any preparation or deliberation. It is possible that these different

learner types have very different learning outcomes or different responses to failure.

Initial analyses could be done with the current data set, however, this would require

more sophisticated data mining techniques.

Exploring triggers of productive persistence

Unfortunately, the current study proved to be uninformative with respect to the

idea of productive persistence. The internal-external EPB affected persistence after

failure but not any particular type of persistence. Moreover, the instrumental behavior

in this study was persistence alone; the type of persistence (at least based on the

current measures) did not have an effect on learning. However, there are certainly

situations where persistence alone is not enough and perseverating strategies do not

work. How can we motivate students to embark on a path of productive persistence?

The current study offers little help in this regard, but prior work with the Teachable

Agents (TA) provides some suggestions for future work. In the TA study, one group

was learning in order to teach their agents, while another group was learning for

themselves, with identical software. The TA students spent far more time on learning

activities, while the Self group squandered their time on game play and chat.

Moreover, the TA group engaged in behaviors (map-building) that we know are

productive for learning in that environment. We speculate that students in the TA

group engaged in productive persistence because they felt a sense of social

responsibility towards their tutees. For example, when the TA fails in the game show,

students engage in many effective learning strategies to remediate their TA’s

knowledge. Whereas, when the students themselves get questions wrong in the game

show (in the Self condition), they do not engage in such effective learning strategies,

and they ultimately learn less.

This explanation assumes that students know the learning strategies and tactics

that are going to be effective, but they choose when to implement them. Beyond

156

Teachable Agents, how can we harness a sense of social responsibility to encourage

productive persistence in students, particularly after failure? Do students know what

productive persistence is or do they lack the SRL and metacognitive knowledge?

It might be interesting to explore these questions in our dream computer-based

learning environment, one where we knew exactly what constitutes productive

persistence. There would be three conditions. In one condition, students would be

given a social responsibility manipulation. They are told that each time they lose a

game, they will lose a “crew member” from their ship, and they can win crew

members back by winning a game. In another condition, students would simply be

told a good SRL strategy that constitutes productive persistence. A third condition

could be given both manipulations. This might begin to answer the question of

whether students need metacognitive guidance to engage in these strategies and

whether this particular instantiation of a sense of responsibility could trigger more

productive persistence.

Conclusion

Providing students with an ego-protective buffer by inviting them to make a

combination of internal and external attributions for failure was a productive

intervention in the context of an educational game. This work has interesting

implications for building a theory of the ego-protective buffer, designing educational

interventions that will help students deal with failure in productive ways, and creating

new measures of functional behaviors. However, more research must be done to fully

understand the phenomenon. This work has raised many interesting questions for

future study in the areas of productive persistence, behavioral measures of motivation

and learning, and the mechanisms by which an internal-external EPB affects

persistence.

157

APPENDIX

Contents of Appendix materials

A. Pretest

B. Posttest

C. Presurveys

D. Postsurvey

E. Example puzzle for each topic

F. Example set of resources for Family Tree topic

158

Appendix A: Pretest

Please work alone and try hard. If you don’t know the answer to a question, give your best guess. Good luck! 1. Have you ever noticed that some people have dimples when they smile?

The gene for dimples has two forms or alleles:

• D = dominant allele for dimples • d = recessive allele for no dimples

(1a) Two dimpled parents have children. Fill in the Punnett square below to show the possible sets of alleles their children could inherit.

Parent 1

D d

D Parent 2

D

(1b) In the Punnett square above, circle the children that will have dimples.

159

2. There are three basic hair types: straight, wavy, and curly. The gene for hair type has two co-dominant alleles: S = co-dominant allele for straight hair C = co-dominant allele for curly hair (2a) Write in the hair type (straight, wavy, or curly) for each missing individual from this family tree. (2b) When you see “probability?”, write in the chances that a child of this hair type would be born from its parents. (Hint: probability is a number.)

160

3. What is the probability that two Dd parents would have….

(3a) a child with genotype DD? _________

(3b) a child with genotype Dd? _________

(3c) a child with genotype dd? _________

(3d) a child with dimples? _________

(3e) a child with no dimples? _________ 4. The family tree below shows the inheritance of two separate traits: hair type and dimples. For each missing individual from this family tree…

(4a) Write in its hair type (straight, wavy, or curly) (4b) Write in its dimple genotype (DD, Dd, or dd) (4c) When you see “probability?” write in the chances that this type of child would be born to its parents.

161

5. A Dd parent and a dd parent plan to have 12 children. (5a) How many DD children would you expect them to have? _________ (5b) How many Dd children would you expect them to have? _________ (5c) How many dd children would you expect them to have? _________ (5d) A DIFFERENT SET of parents has 5 children. All the children are genotype Dd. Predict the most likely genotypes of these parents.

Parent 1 genotype ________ Parent 2 genotype ________ (5e) There are OTHER sets of parents that could have 5 Dd children, though it would be less likely. What is one possible set? (It’s OK to use one of the parents from 5d).

Parent 1 genotype ________ Parent 2 genotype ________ 6. The gene for dimples comes in two forms:

D = dominant allele for dimples d = recessive allele for no dimples

The gene for hair type comes in two co-dominant forms:

S = co-dominant allele for straight hair C = co-dominant allele for curly hair

Suppose the following two parents plan to have children: DdSC and DDSC. (6a) Fill in the Punnett square below to show the possible sets of alleles these parents could pass on to their children. DS dS DC dC

DS

DC

(6b) In the Punnett square above, circle the children that would have both dimples and wavy hair.

162

(6c) What is the probability that these parents would have a DDSC child? ___________ (6d) What is the probability that these parents would have a DdCC child? ___________ 7. The family tree below traces the inheritance of dimples. (7a) Write in the genotype (DD, Dd, or dd) for each missing individual in this family tree. (7b) When you see “probability?”, write in the chances that this type of child would be born from its parents. (Hint: probability is a number.)

163

Appendix B: Post-test

Please work alone and try hard. If you don’t know the answer to a question, give your best guess. Good luck! 1. Some lizards have different coat patterns.

The gene for coat pattern has two forms or alleles:

• R = dominant allele for spotted coat • r = recessive allele for plain coat

(1a) Two spotted lizard parents have children. Fill in the Punnett square below to show the possible sets of alleles their children could inherit.

Parent 1

R r

R Parent 2

r

(1b) In the Punnett square above, circle the children that will have spotted coats.

164

2. There are three lizard tail lengths: short, medium, and long. The gene for tail length has two co-dominant alleles: L = co-dominant allele for long tail S = co-dominant allele for short tail (2a) Write in the tail type (short, medium, or long) for each missing individual from this family tree. (2b) When you see “probability?”, write in the chances that a child of this tail type would be born from its parents. (Hint: probability is a number.)

165

3. What is the probability that two Rr parents would have….

(3a) a child with genotype RR? _________

(3b) a child with genotype Rr? _________

(3c) a child with genotype rr? _________

(3d) a child with a spotted coat? _________

(3e) a child with a plain coat? _________ 4. The family tree below shows the inheritance of two separate traits: tail type and coat pattern. For each missing individual from this family tree…

(4a) Write in its tail type (short, medium, or long) (4b) Write in its coat pattern genotype (RR, Rr, or rr) (4c) When you see “probability?” write in the chances that this type of child would be born to its parents.

166

5. An Rr parent and an rr parent plan to have 12 children. (5a) How many RR children would you expect them to have? _________ (5b) How many Rr children would you expect them to have? _________ (5c) How many rr children would you expect them to have? _________ (5d) A DIFFERENT SET of parents has 5 children. All the children are genotype Rr. Predict the most likely genotypes of these parents.

Parent 1 genotype ________ Parent 2 genotype ________ (5e) There are OTHER sets of parents that could have 5 Rr children, though it would be less likely. What is one possible set? (It’s OK to use one of the parents from 5d).

Parent 1 genotype ________ Parent 2 genotype ________ 6. The gene for coat pattern comes in two forms:

R = dominant allele for spotted coat r = recessive allele for plain coat

The gene for tail type comes in two co-dominant forms:

L = co-dominant allele for long tail S = co-dominant allele for short tail

Suppose the following two parents plan to have children: RrLS and RRLS. (6a) Fill in the Punnett square below to show the possible sets of alleles these parents could pass on to their children. RL rL RS rS

RL

RS

(6b) In the Punnett square above, circle the children that would have both spots and medium tails.

167

(6c) What is the probability that these parents would have a RRLS child? ___________ (6d) What is the probability that these parents would have a RrSS child? ___________ 7. The family tree below traces the inheritance of coat pattern. (7a) Write in the genotype (RR, Rr, or rr) for each missing individual in this family tree. (7b) When you see “probability?”, write in the chances that this type of child would be born from its parents. (Hint: probability is a number.)

168

8. Ms. Schmidt is holding two raffles: one for a cupcake and one for a cookie. Students have purchased tickets for these raffles. Ms. Schmidt puts the raffle tickets into 2 separate bowls. She stirs up the tickets and picks one winner from each bowl. What is the probability that Sarah will win both the cupcake and the cookie? Show your work. 9. In sorghum plants, red stem is dominant over green stem.

• R=dominantalleleforredstem• r=recessivealleleforgreenstem

1,000 seeds from 2 parent sorghum plants grow to become 759 red-stemmed plants and 241 green-stemmed plants. Which genotypes are the plant parents most likely to have? Explain why. 10. The alleles for fish egg coat are:

• T = dominant allele for thin egg coat • t = recessive allele for thick egg coat

Raffle tickets

169

A rare breed of fish cannot be born with two recessive alleles for thick egg coat. A pair of recessive egg coat alleles is deadly because it prevents the fish from hatching from its egg. (10a) Fill in the Punnett square below to show all the possible sets of alleles that 2 parent Tt fish could pass on to their children.

T t

T

t

(10b) In the Punnett square above, circle the genotype(s) that cannot be born. (10c) What is the probability that a TT fish will be born to 2 Tt parents? 11. Your friend plays a card game called Stanford solitaire. There are 4 kinds of Stanford solitaire cards:

You watch your friend play the game and try to figure out the rules. Here’s what she does:

1. At the beginning of each round, she deals herself four cards and places them in two different groups, face down on the table. She does not look at the cards.

2. She randomly picks one card from each group and places each one face up on

the table. These cards are called the “pair.”

170

3. She calls the round a “win” or a “loss”. Below are some winning and losing pairs.

(11a) What is the rule for winning a round? What kind of pair wins a round?

(11b) What is the rule for losing a round? What kind of pair loses a round?

171

(11c) Which of the following group cards would have the greatest chances of producing a win?

a.

b.

c.

d. Options (a) and (c) have the same chance. e. Options (a) and (b) have the same chance.

172

Appendix C: Pre-surveys

Motivational Constructs

Tell us how much you agree with the following sentences by circling one number for each. Please give us your honest opinion for each statement. There are no right or wrong answers, and your teacher will not see them.

Stro

ngly

D

isag

ree

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

A

gree

1. An important reason I do my science work is because I like to learn new things. 1 2 3 4 5

2. What I learn in my science class is interesting. 1 2 3 4 5

3. If I could choose an easy or hard science assignment, I would choose an easy one. 1 2 3 4 5

4. I am certain I can figure out how to do difficult work in science. 1 2 3 4 5

5. I like the work in my science class best when it really makes me think. 1 2 3 4 5

6. I would feel successful in science if I got better grades than most of the other students. 1 2 3 4 5

7. How well I do in science depends on how smart I was when I was born. 1 2 3 4 5

8. I enjoy the activities in my science class. 1 2 3 4 5

9. I can do almost all the work in science class if I do not give up. 1 2 3 4 5

10. I have to be really smart to do well in science.

11. I want to do better than other students in my science class. 1 2 3 4 5

12. An important reason I do my science work is so other students won’t think I’m stupid. 1 2 3 4 5

13. I like science work that I will learn from even if it is hard to learn at first. 1 2 3 4 5

14. If I have enough time, I can do a good job on all my science work. 1 2 3 4 5

15. It is important to me that the other students in my 1 2 3 4 5

173

Stro

ngly

D

isag

ree

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

A

gree

science class think I am smart.

16. I find science interesting. 1 2 3 4 5

17. It is very important to me that I do not look stupid in my science class. 1 2 3 4 5

18. I feel most successful in science class when I learn something I didn’t know before. 1 2 3 4 5

19. I am certain I can master the skills taught in science class this year. 1 2 3 4 5

20. I don’t like doing difficult science work because I might fail. 1 2 3 4 5

21. I can’t change how smart I am in science 1 2 3 4 5

22. I like learning about science concepts. 1 2 3 4 5

23. I like to do science work that lets me show how smart I am. 1 2 3 4 5

Thank you very much!

174

Self-regulated learning skills survey

Instructions: Please circle the number that best describes what you think about each sentence.

Not

at a

ll tru

e So

mew

hat

true

Ver

y tru

e

24. I ask myself questions to make sure I know the science material I have been studying. 1 2 3 4 5 6 7

25. When I study for a science test, I try to put together the information from class and from the book.

1 2 3 4 5 6 7

26. I work on practice science exercises and answer end of chapter questions even when I don’t have to.

1 2 3 4 5 6 7

27. When I do science homework, I try to remember what the teacher said in class so I can answer the questions correctly.

1 2 3 4 5 6 7

28. Before I begin studying for science I think about the things I will need to do to learn. 1 2 3 4 5 6 7

29. When I study for science I put important ideas into my own words. 1 2 3 4 5 6 7

30. When I study for a science test, I try to remember as many facts as I can. 1 2 3 4 5 6 7

31. I work hard to get a good grade even when I don’t like science. 1 2 3 4 5 6 7

32. I use what I have learned from old homework assignments and the textbook to do new science assignments.

1 2 3 4 5 6 7

33. When I am solving a science problem, I stop once in a while to go over what I have done. 1 2 3 4 5 6 7

34. Even when my science work is dull and boring, I keep working until I finish. 1 2 3 4 5 6 7

35. I outline the chapters of my science book to help me study. 1 2 3 4 5 6 7

36. I often find that I have been doing all the work for science class but don’t know what it is all about.

1 2 3 4 5 6 7

175

Not

at a

ll tru

e So

mew

hat

true

Ver

y tru

e

37. I always try to understand what my science teacher is saying, even if it doesn’t make sense. 1 2 3 4 5 6 7

38. When I read for science class, I try to connect the things I am reading about with what I already know.

1 2 3 4 5 6 7

39. When my science work is hard, I either give up or study only the easy parts. 1 2 3 4 5 6 7

176

Appendix D: Post-survey

Think back to the genetics game you played recently. How much do you agree with the following statements about the game? Circle one answer for each.

Dis

agre

e a

lot

Dis

agre

e a

little

Neu

tral

Agr

ee a

lit

tle

Agr

ee

a lo

t

40. I enjoyed playing the game. 1 2 3 4 5

41. I was skilled at playing the game. 1 2 3 4 5

42. I would like to play another game like this, if I could. 1 2 3 4 5

43. I was successful at playing the game. 1 2 3 4 5

44. The game was fun. 1 2 3 4 5

45. I learned from the game. 1 2 3 4 5

46. Imagine that your friend Tai plays a puzzle. He gets 7 out of 10 challenges

right and loses the puzzle. Why was this game lost? 47. Imagine that your friend Tai plays a different puzzle. This time, he gets 9 out

of 10 challenges and wins the puzzle. Why was this game won?

9. Did you have a strategy for playing the game? Tell us about it.

Thank you very much! We really appreciate your help!

177

Appendix E: Example Puzzle for each Topic

Punnett squares

178

Probability

179

Prediction

180

Family Trees

181

Co-dominance

182

Punnett squares with 2 genes

183

Family trees with 2 genes

184

Appendix F: Example Set of Resources for Family Tree Topic

Preview

The preview is a practice puzzle. Students can input answers but they will not get

“graded.”

185

Reading

186

Feedback

187

Answers

188

Explanation of Answers

189

References

Aleven, V., & Koedinger, K. R. (2000). Limitations of student control: Do students

know when they need help? In C. F. G. Gauthier & K. VanLehn (Eds.), Proceedings of the 5th International Conference on Intelligent Tutoring Systems, ITS 2000 (pp. 292–303). Berlin: Springer-Verlag.

Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2004). Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. In J. C. Lester, R. M. Vicario, & F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems (pp. 227-239). Berlin: Springer Verlag.

Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73(3), 277-320.

Allport, G.W. (1937). Personality: A psychological interpretation. New York: Holt. Ames, C. & Archer, J. (1988). Achievement goals in the classroom: Students’

learning strategies and motivation processes. Journal of Educational Psychology, 80(3), 260-267.

Azevedo, R., Guthrie, J.T., & Seibert, D. (2004). The role of self-regulated learning in fostering students’ conceptual understanding of complex systems in hypermedia. Journal of Educational Computing Research, 30(1&2), 87-111.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman.

Baker, R.S., Corbett, A.T., & Koedinger, K.R. (2004). Detecting student misuse of intelligent tutoring systems. Lecture Notes in Computer Science, 3220, 54-76.

Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z. (2004) Off-Task Behavior in the Cognitive Tutor Classroom: When Students "Game The System". Proceedings of ACM CHI 2004: Computer-Human Interaction, 383-390.

Bereiter, C., & Scardamalia, M. (1992). Surpassing ourselves: An inquiry into the nature and implications of expertise. Chicago: Open Court.

Biswas, G., Leelawong, K., Schwartz, D. L., Vye, N., & TAG-V (2005). Learning by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363-392.

Blair, K. P. (2009). The neglected importance of feedback perception in learning: An analysis of children and adult’s uptake of quantitative feedback in a mathematics simulation environment. Unpublished dissertation.

Blumenfeld, P.C., Pintrich, P.R., Meece, J., & Wessels, K. (1982). The formation and role of self perceptions of ability in elementary classrooms. The Elementary School Journal, 82(5), 400-420.

Butler, D.L., & Winne, P.I. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245-281.

Chase, C.C. (In press). Motivating expertise: Equipping novices with the motivational tools to move beyond failure. To appear in J.J. Staszewski (Ed.), Expertise and

190

skill acquisition: The impact of William G. Chase. New York: Psychology Press.

Chase, C.C., Chin, D.B., Oppezzo, M.A., & Schwartz, D.L. (2009). Teachable Agents and the Protégé Effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18(4), 334-352.

Chase, W.G., & Simon, H.A. (1973). The mind’s eye in chess. In W.G. Chase (Ed.), Visual information processing (pp. 215-281). New York: Academic.

Chin, D.B., Dohmen, I.M., Cheng, B.H., Oppezzo, M.A., Chase, C.C., & Schwartz, D.L. (2010). Preparing students for future learning with Teachable Agents. Educational Technology Research and Development, 58(6), 649-669.

Covington, M.V., & Omelich, C.L. (1981). As failures mount: Affective and cognitive consequences of ability demotion in the classroom. Journal of Educational Psychology, 73(6), 796-808.

Deci, E.L., & Ryan, R.M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press.

Dunbar, K. (1999). How scientists build models: In vivo science as a window on the scientific mind. In L. Magnani, N. Nersessian, & P. Thagard (Eds.), Model-based reasoning in scientific discovery (pp. 89-98). New York: Plenum Publishers.

Diener C. & Dweck, C. (1978). An analysis of learned helplessness: Continuous changes in performance, strategy, and achievement cognitions following failure. Journal of Personality and Social Psychology, 36, 451-462.

Dweck, C.S. (1986). Motivational processes affecting learning. American psychologist, 41(10), 1040-1048.

Ericsson, K.A. (2002). Attaining excellence through deliberate practice: Insights from the study of expert performance. In M. Ferrari (Ed.), The pursuit of excellence through education (pp. 21-55). Mahwah, New Jersey: Lawrence Erlbaum Associates.

Elig, T.W., & Frieze, I.H. (1979). Measuring causal attributions for success and failure. Journal of Personality and Social Psychology, 37(4), 621-634.

Ertmer, P.A., & Newby, T.J. (1996). The expert learner: Strategic, self-regulated, and reflective. Instructional Science, 24(1), 1-24.

Frieze, I.H., & Bar-Tal, D. (1980). Developmental trends in cue utilization for attributional judgments. Journal of Applied Developmental Psychology, 1(2), 83-94.

Hiroto, D.S., & Seligman, M.E. (1975). Generality of learned helplessness in man. Journal of Personality and Social Psychology, 31(2), 311-327.

Kapur, M. (2008.) Productive failure. Cognition and Instruction, 26, 379-424.

Kluger, A.N., & DeNisi, A. (1996). Effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284.

191

Lau, S., & Roeser, R. W. (2002). Cognitive abilities and motivational processes in high school students' situational engagement and achievement in science. Educational Assessment, 8(2), 139-162.

Leddo, J., Abelson, R.P., & Gross, P.H. (1984). Conjunctive explanations: When two reasons are better than one. Journal of Personality and Social Psychology, 47(5), 933-943.

Mandl, H., Gräsel, C., & Fischer, F. (2000). Problem-oriented learning: Facilitating the use of domain-specific and control strategies through modeling by an expert. In W. J. Perrig & A. Grob (Eds.), Control of human behavior, mental processes and consciousness (pp. 165–182). Mahwah, NJ: Erlbaum.

Mezulis, A.H., Abramson, L.Y., Hyde, J.S., Hankin, B.L. (2004). Is there a universal positivity bias in attributions? A meta-analytic review of individual, developmental, and cultural differences in the self-serving attributional bias. Psychological Bulletin, 130(5), 711-747.

Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., Gheen, M., Kaplan, A., Kumar, R., Middleton, M. J., Nelson, J., Roeser, R., & Urdan, T., (2000). Manual for the Patterns of Adaptive Learning Scales (PALS), Ann Arbor, MI: University of Michigan.

Nicholls, J. (1990). What is ability and why are we mindful of it? A developmental perspective. In R. Sternberg & J. Kolligian, Jr. (Eds.). Competence considered (pp. 11-40). New Haven: Yale University Press.

Nelson-Le Gall, S. (1981). Help-seeking: An understudied problem-solving skill in children. Developmental Review, 1, 224–246.

Newman, R. S. (1994). Adaptive help seeking: A strategy of self-regulated learning. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulation of learning and performance: Issues and educational applications (pp. 283–301). Hillsdale, NJ: Erlbaum.

Nisbett, R.E. (1987). Teaching reasoning. Science, 238(4827), 625-631. Perry, N.E. (1998). Young children’s self-regulated learning and contexts that support

it. Journal of Educational Psychology, 90, 715-729. Peterson, C., Semmel, A., von Baeyer, C., Abramson, L.Y., Metalsky, G., &

Seligman, M. (1982). The attributional style questionnaire. Cognitive Therapy and Research, 6(3), 287-299.

Peterson, C., Maier, S.F., & Seligman, M.E. (1993). Learned helplessness: A theory for the age of personal control. New York: Oxford University Press.

Pintrich, P.R. (2004). A conceptual framework for assessing motivation and self-regulated learning in college students. Educational Psychology Review, 16(4), 385-407.

Pintrich, P.R., Smith, D.A.F., Garcia, T. & Mckeachie, W.J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (Mslq). Educational Psychology and Measurement, 53(3), 801-813.

192

Pintrich, P.R., & De Groot, E.V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1) 33-40.

Relich, J.D., Debus, R.L., & Walker, R. (1986). The mediating role of attribution and self-efficacy variables for treatment effects on achievement outcomes. Contemporary Educational Psychology, 11(3), 195-216.

Reeve, J.M., Olson, B.C., & Cole, S.G. (1985). Motivation and performance: Two consequences of winning and losing in competition. Motivation and Emotion, 9(3), 291-298.

Renkl, A. (2002). Learning from worked-out examples: Instructional explanations supplement self-explanations. Learning & Instruction, 12, 529–556.

Rhodewalt, F., Morf, C.C., Hazlett, S., & Fairfield, M. (1991). Self-handicapping: The role of discounting and augmentation in the preservation of self-esteem. Journal of Personality and Social Psychology, 61, 122-31.

Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2007). Can help seeking be tutored? Searching for the secret sauce of metacognitive tutoring. In R. Luckin, K. R. Koedinger, & J. Greer (Eds.), Proceedings of the international conference on artificial intelligence in education (pp. 203-210). Amsterdam: IOS Press.

Santagata, R. (2005) Practices and Beliefs in Mistake-Handling Activities: A Video Study of Italian and U.S. Mathematics Lessons. Teaching and Teacher Education, 21, 491-508.

Schlenker, B.R., Pontari, B.A., Christopher, A.N. (2001). Excuses and character: Personal and social implications of excuses. Personality and Social Psychology Review, 5(1), 15-32.

Schunk, D.H. (1982). Effects of effort attributional feedback on children’s perceived self-efficacy and achievement. Journal of Educational Psychology, 74, 548-556.

Schunk, D. H. (1983). Ability versus effort attributional feedback: Differential effects on self-efficacy and achievement. Journal of Educational Psychology, 75, 848-856.

Schunk, D.H., Pintrich, P.R., & Meece, J.L. (2008). Motivation in education: Theory, research, and applications. Upper Saddle River: Pearson/Merrill Prentice Hall.

Snyder, C.R., & Higgins, R.L. (1988). Excuses: Their effective role in the negotiation of reality. Psychological Bulletin, 104(1), 23-35.

Schwartz, D. L., Blair, K. P., Biswas, G., Leelawong, K., & Davis, J. (2007). Animations of thought: Interactivity in the teachable agents paradigm. In R. Lowe & W. Schnotz (Eds). Learning with animation: Research and implications for design (pp. 114-40). UK: Cambridge University Press.

Schwartz, D.L., & Martin, T. (2004). Inventing to Prepare for Future Learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129-184.

Seligman, M., & Maier, S. (1967). Failure to escape traumatic shock. Journal of Experimental Psychology, 74, 1-9.

193

Shute, V. J., & Gluck, K. A. (1996). Individual differences in patterns of spontaneousonline tool use. Journal of the Learning Sciences, 5, 329–355.

Siegler, R.S. (2002). Microgenetic studies of self-explanation. In N. Granott & J. Parziale (Eds). Microdevelopment: Transition processes in development and learning (pp. 31-58). UK: Cambridge University Press.

Stigler, J. W. & Hiebert, J. (1999). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York: The Free Press.

Stipek, D.J., & Weisz, J.R. (1981). Perceived personal control and academic achievement. Review of Educational Research, 51(1), 101-137.

Taylor, S.E., & Brown, J.D. (1988). Illusion and well-being: A social psychological perspective on mental health. Psychological Bulletin, 103(2), 193-210.

Turner, J.C. (1995). The influence of classroom contexts on young children’s motivation for literacy. Reading Research Quarterly, 30, 410-441.

Vanlehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209-249.

Vallerand, R.J., Gauvin, L.I., & Halliwell, W.R. (1986). Effects of zero-sum competition on children’s intrinsic motivation and performance. The Journal of Social Psychology, 126(4), 465-472.

Weiner, B. (1979). A theory of motivation for some classroom experiences. Journal of Educational Psychology, 71(1), 3-25.

Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548-573.

Weiner, B. (1986). An attributional theory of motivation and emotion. New York: Springer-Verlag.

Weinstein, C.E., Schulte, A., & Palmer, D. (1987). LASSI: Learning and study strategies inventory. Clearwater, FL: H&H Publishing.

Winne, P.I. (2001). Self-regulated learning viewed from models of information processing. In B.J. Zimmerman & D. Schunk (Eds). Self-Regulated Learning and Academic Achievement: Theoretical Perspectives. (pp. 153-190). Mahwah, NJ: Lawrence Erlbaum Associates.

Winne, P.I., & Perry, N.E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds). Handbook of Self-Regulation. (pp. 531-566). San Diego: Academic Press.

Wood, D. (2001). Scaffolding, contingent tutoring, and computer-supported learning. International Journal of Artificial Intelligence in Education, 12, 280–292.

Wood, H., & Wood, D. (1999). Help seeking, learning and contingent tutoring. Computers and Education, 33, 153–169.

Zimmerman, B.J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329-339.

Zimmerman, B.J. (1994). Dimensions of academic self-regulation: A conceptual framework for education. In D.H. Schunk & B.J. Zimmerman (Eds). Self-regulation of Learning and Performance: Issues and Educational Applications. (pp. 3-24). Hillsdale, NJ: Lawrence Erlbaum Associates.

194

Zimmerman, B.J. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64-70.

Zimmerman, B.J., & Marinez-Pons, M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23, 614-628.