Internet and Higher Education 15 (2012) 89–95

Contents lists available at SciVerse ScienceDirect

Internet and Higher Education

Learning presence: Additional research on a new conceptual element within theCommunity of Inquiry (CoI) framework

Peter Shea a,⁎, Suzanne Hayes b, Sedef Uzuner Smith f, Jason Vickers a, Temi Bidjerano c, Alexandra Pickett d,Mary Gozza-Cohen e, Jane Wilde a, Shoubang Jian a

a University at Albany — SUNY, United Statesb Empire State College — SUNY, United Statesc Furman University, United Statesd SUNY Learning Network, United Statese University at Albany, United Statesf Indiana University of Pennsylvania, United States

⁎ Corresponding author.E-mail address: pshea@uamail.albany.edu (P. Shea).

1096-7516/$ – see front matter © 2011 Elsevier Inc. Alldoi:10.1016/j.iheduc.2011.08.002

a b s t r a c t

a r t i c l e i n f o

Available online 26 August 2011

Keywords:Community of InquiryOnline learningLearner presenceSelf regulated learning

This paper presents an empirical study grounded in the Community of Inquiry framework (Garrison, AndersonArcher, 2000) and employs quantitative content analysis of student discourse and other artifacts of learning inonline courses in an effort to enhance and improve the framework and offer practical implications for onlineeducation. As a theoretical framework the purpose of the widely referenced CoI model is to describe, explain,and predict learning in online environments. The current study grows out of an ongoing research agenda tounderstand student and faculty experiences in emerging technology-mediated education systems and to makerecommendations for theory and practice. The major question addressed here is whether the CoI modeladequately explains effective learner behavior in fully online courses and to articulate a new conceptual element— learning presence. Results indicate that learning presence is evident in more complex learning activities thatpromote collaboration and is correlated with course grades.

rights reserved.

© 2011 Elsevier Inc. All rights reserved.

1. Introduction

Online learning in U.S. higher education has exploded in recentyears. Studies by the U.S. Department of Education (Parsad & Lewis,2008) indicate that online students generated more than 12 millioncourse enrollments in 2007–2008 with nearly thirty percent of Amer-ican college students enrolled in at least one online course (Allen &Seaman, 2010). For learners who were married with dependent chil-dren that percentage rises to one in three (Staklis, 2010). Given thattoday's growth in distance higher education continues to be drivenlargely by developments in asynchronous online learning (Allen &Seaman, 2008; Parsad & Lewis, 2008; U. S. Department of Education.& National Center for Education Statistics., 2008) it is necessary thatwe focus our attention on these rapidly growing environments.

Recent large scale meta-analytic research indicates that successrates for online students are at least equivalent (Allen, Bourhis, Burrell,&Mabry, 2002; Bernard et al., 2004; Tallent-Runnels et al., 2006; Zhao,Lei, Yan, Lai, & Tan, 2005) and may be better than (Means, Toyama,Murphy, Bakia, & Jones, 2009) those of classroom students. Zhaoet al. (2005) investigated the “heterogeneity” of previous empirical

results and began to identify the conditions under which distanceand online education resulted in better outcomes (Zhao et al.,2005). A telling condition was “publication year” with an increasingnumber of studies after 1998 revealing advantages for the online for-mat. Zhao et al. concluded that this finding suggested that the two-way interaction of Internet-based online applications of distancelearning provided advantages that previous technological affor-dances had not. Zhao et al. also concluded that studies in which in-structor interaction with online students was medium to highresulted in better learning outcomes for online students relative toclassroom learners.

Means et al. (2009) similarly conducted a comprehensive meta-analysis including an exhaustive search of 1132 studies that com-pared online and face-to-face conditions. The research team exam-ined these to locate the most rigorous studies employing onlyexperimental and quasi-experimental research designs (Meanset al., 2009). Applying these criteria the authors conducted an analy-sis of the 56 most rigorous studies of online education. Reviewingstudies that investigated elements of online learner self-regulation(e.g., Bixler, 2008; Chang, 2007; Chung, Chung, & Severance, 1999;Cook, Dupras, Thompson, & Pankratz, 2005; Crippen & Earl, 2007;Nelson, 2007; Saito & Miwa, 2007; Shen, Lee, & Tsai, 2007; Wang,Wang, Wang, & Huang, 2006) the authors found advantageous out-comes for scaffolding learning strategies including self-reflection,

90 P. Shea et al. / Internet and Higher Education 15 (2012) 89–95

self-explanation, and self-monitoring. These positive findings foronline learner self-regulation represent fertile ground for the devel-opment of a more comprehensive explanatory model for understand-ing the potential benefits of online instruction, a task to which wenow turn.

1.1. Theoretical perspectives

The present study is grounded in several complementary theoret-ical perspectives reflective of research on the complexity of inter-action in online classrooms. Rex, Steadman and Graciano (2006)suggest that such inquiry falls within seven traditions and thisstudy contains elements of several of these including process-product, cognitive–constructivist, socio-cognitive, and situative per-spectives. From a process–product perspective we are interestedin how online discourse informs and shapes jointly created productsdeveloped by collaborative teams. This study also attempts to bridgecognitive and socio-cognitive/situative perspectives by introducingconcepts of learner self-regulation to the Community of Inquiryframework (Garrison, Anderson, & Archer, 2000).

1.2. Community of Inquiry model and self-regulation

The CoI framework is based on a model of critical thinking andpractical inquiry. The authors posit that learning occurs through theinteraction of students and their instructor and is manifest as threeintegrated elements that contribute to a successful online learningcommunity: social presence (SP), teaching presence (TP), and cogni-tive presence (CP). The framework theorizes online knowledgebuilding as the outcome of collaborative work among active par-ticipants in learning communities reflecting instructional orchestra-tion appropriate to the online environments (teaching presence)and an encouraging collegial online setting (social presence). Theteaching presence construct delineates task sets such as organization,design, discourse facilitation, and direct instruction (Anderson,Rourke, Garrison, & Archer, 2001) and articulates the behaviors likelyto result in a productive Community of Inquiry (e.g., Shea, Li, Swan, &Pickett, 2005). Social presence represents online discourse that pro-motes positive affect, interaction, and cohesion (Rourke, Anderson,Garrison, & Archer, 1999) that support a functional collaborative en-vironment. Themodel also includes cognitive presence, a multivariatemeasure of critical and creative thinking that results from the cyclicalprocess of practical inquiry within such a community of learners. Thespecific form of interaction within the cognitive presence constructthus reflects a pragmatic view of learning (Dewey, 1933; Lipmann,2003, Pierce, 1955).

Given that learner self-regulation is a well researched constructcompatible with multiple theories of learning (see e.g., Zimmerman,2000, 2008) its introduction has the potential to enhance the CoIframework, which foregrounds the social construction of knowledgewith less focus on the self- and co-regulatory strategies that success-ful individual learners may employ. We propose that the introductionof learner self-regulation can better enhance the scope of the CoIframework.

Recent work on the CoI model (Shea, Hayes, & Vickers, 2010) sug-gested that previous research methods may have resulted in a sys-tematic underrepresentation of the instructional effort involved inonline education. Using quantitative content analysis these authorsexamined course documents within and external to threaded discus-sion areas and concluded that the majority of teaching presence intwo undergraduate online business courses occurred outside ofthreaded discussion (emails, assessments, private folders, etc.),areas that are generally not included in past investigations of theteaching presence construct (e.g., Akyol & Garrison, 2008; Akyol &Garrison, 2011, Anderson et al., 2001; Bliss & Lawrence, 2009; Coll,Engel & Bustos, 2009; Pawan, Paulus, Yalcin & Chang, 2003). This

ongoing project to document all instances of teaching, social, and cog-nitive presence in complete online courses also resulted in identifica-tion of learner discourse that did not fit within themodel, i.e. could notbe reliably coded as indicators of teaching, social, or cognitive pres-ence (Shea & Bidjerano, 2010; Shea, Hayes, & Vickers, 2010). These ex-ceptions represent interesting data for refining and enhancing themodel as they suggest that learners are attempting to accomplishgoals that are not accounted for within the CoI framework. This re-search concluded that the learners under investigation engaged indiscourse on course logistics including collaborative attempts to un-derstand instructions provided to them by the course professor.Learner discussions also included strategic efforts to divide up tasks,manage time, and set goals in order to successfully complete groupprojects. As such they appeared to be indicators of online learnerself- and co-regulation, which can be viewed as the degree to whichstudents in collaborative online educational environments are meta-cognitively, motivationally, and behaviorally active participants inthe learning process (Zimmerman, 1986).

1.3. Self-regulated learning and learning presence

Research on self-regulated learning indicates that, “it is viewed asespecially important during personally directed forms of learning,such as discovery learning, self-selected reading, or seeking informa-tion from electronic sources, [but is] also deemed important in socialforms of learning” (Zimmerman, 2008). Given the electronic, social,and self-directed nature of online learning, it seems imperative thatwe examine learner self- and co-regulation in online environmentsespecially as they relate to desired outcomes such as higher levelsof critical and creative thinking as described in the CoI framework.Accomplishing this goal requires that we examine a wide variety ofissues including meta-cognitive, motivational, and behavioral traitsand activities that are under the control of successful online learnersand which past research indicates may be fostered in online environ-ments (Means et al., 2009). We suggest that this constellation ofbehaviors and traits may be seen as elements of a larger construct“learning presence” (Shea & Bidjerano, 2010). We suggest that thename “learning presence” integrates with the other forms of presencein the CoI framework and reflects the proactive stance adopted bystudents who marshal thoughts, emotions, motivations, behaviorsand strategies in the service of successful online learning. Learningpresence thus indicates the exercise of agency and control ratherthan compliance and passivity and more fully articulates popular be-liefs about the importance of self-direction in online environments.

This articulation of online learning presence is in no way meant toundermine or replace the shared instructional roles of teachersand learners described in the teaching presence construct ofthe CoI framework. Our goal is to examine the distinct roles that suc-cessful online learners may adopt, roles that do not apply to instruc-tors per se. While we recognize that recent researchers (e.g. Akyol& Garrison, 2011) have expressed concern that a construct that sep-arates the role of teacher and learner may undermine the collabora-tive focus of the CoI model, we believe that the opposite may alsobe a danger. We suggest that while considerable overlap exists be-tween the roles of online instructors and learners their duties, expec-tations, motivations, and mechanisms for success are not identical.Further articulating the strategies and activities of successful onlinelearners will, we hope, enhance rather than diminish the explanatorypower of the CoI model.

2. Method and materials

In previous research (Shea, Hayes, & Vickers, 2010) student dis-course that occurred in certain collaborative activities could not be re-liably coded as teaching, social, or cognitive presence. Given thatthese activities are core to learner-centered approaches to online

5

Average Student LPCombined Debate Prep. v. Debate Discussion

Course B

91P. Shea et al. / Internet and Higher Education 15 (2012) 89–95

education we felt it essential to integrate them into the CoI frame-work. This required that we look for additional theoretical groundingto attempt to understand and categorize the discourse. We usedconcepts from a variety of sources (Azevedo, Cromley, & Seibert,2004a; Azevedo, Guthrie, & Seibert, 2004b; Curtis & Lawson, 2001;Zimmerman, 1989, 2008) that examined self-regulated learning andcollaboration to accomplish this goal. Using conceptual elementfrom these sources we then searched for patterns of self- and co-regulation within areas of the course where previous attempts tocode student discourse using standard CoI indicators proved unreli-able. This discourse included student small-group debate preparationareas, “ask a question” areas as well as within two of the full-class dis-cussions in two courses. As a result of this exploratory analysis, wedeveloped a coding scheme that represents what we call learningpresence (see Appendix A).

Once the coding scheme was established, two researchers ana-lyzed all of the communicative processes in the two courses, includ-ing all six discussions, a separate debate discussion and four debatepreparation areas, as well as “ask a question” areas for both coursesections. The goal of this analysis was to determine if the theoryderived codes indicating — self- and co-regulatory behaviors wouldbe evident in online learner discourse. The researchers met to com-pare results of coding and established an inter-rater reliability metricusing Cohen's kappa and Holsti's coefficient of reliability. Cohen'skappa measures “the achieved beyond-chance agreement as a pro-portion of the possible beyond-chance agreement” (Sim & Wright,2005, p.258). A limitation of Cohen's kappa is that symmetrical im-balance in the marginal distribution of agreements can result in asignificantly lower kappa (Feinstein & Cicchetti, 1990). In orderto compensate for this, we employed Holsti's coefficient of reliability(Holsti's CR), which reports the simple agreement of raters.Lombard, Snyder-Duch, and Bracken (2002) suggest utilizing multi-ple reliability indices when interpreting inter-rater reliability, andpast CoI research have utilized this approach (see Garrison, et al.,2001; Rourke et al., 1999; Shea, Hayes, & Vickers, 2010; Shea et al.,2010). Initial and negotiated inter-rater agreement was establishfor each coding session. The benefits of this approach are that coderscan locate transcription errors as well as refine the coding scheme.

When the patterns had been identified, we collated the data bycategories. From these data we made charts indicating the numberof occurrences for our major categories: forethought/planning, moni-toring, and strategy use.

In addition to identifying instances of learning presence we alsoused data generated in previous phases of the research to attemptto understand the relationship between learning presence and theother elements within the CoI model as well as learning outcomes.To accomplish this we examined correlations between teachingpresence, social presence, cognitive presence, learning presence andcourse grades.

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

All Prep Areas Debate Discussion

FP MO SU

Fig. 1. Forethought (FP), monitoring (MO) and strategy use (SU) in different onlineactivities.

3. Results

3.1. Analysis of inter-rater reliability

Tables B1 and B2 in Appendix B reflects the initial and negotiatedinter-rater reliability for both Instructor A and Instructor B. Initialkappa for Instructor A was 0.248–0.640, and initial CR was 0.600–1.000. Initial kappa for Instructor B ranged from 0.054 to 0.834, andinitial CR ranged from 0.902 to 1.000. Discussion IRR can be seen inTable 2 (see Appendix B). Initial kappa for Instructor A was 0.217–0.648, and initial CR was 0.837–0.957. Initial kappa for InstructorB ranged from 0.309 to 0.746, and initial CR ranged from 0.887to 0.976. Variation in kappas may have been a result of symmetricimbalance of no codes for LP indicators. As reflected by Holsti's CR,however, inter-rater agreement is well above the recommended

0.70 IRR for exploratory research of this nature (see Lombard et al.,2002).

3.2. Analysis of learning presence by learning activity and module

Learning presence appears to have different components(forethought/planning, monitoring, and strategy use) that are con-textually dependent. For example, all three are evident in debatepreparation areas, but only strategy use appears in actual discussions.It appears that self- and co-regulation are triggered by the kinds ofactivities that learners are asked to complete. Fig. 1 provides a visualrepresentation of the levels of learning presence that are evident intwo illustrative learning activities.

As can be seen there is forethought/planning, monitoring andstrategy use in the areas where students were divided into smallgroups and were instructed to prepare for a class debate by authoringa position paper. This contrasts with the full class debate discussion inwhich only monitoring behavior was demonstrated.

3.3. Analysis of learning presence and other forms of presence

Figs. 2 and 3 also indicate that learningpresence varies by the type ofactivity students are asked to complete. Fig. 2 shows that while there isa relative paucity of learner self- and co-regulation in traditional activi-ties such as standard threaded discussions, learning presence is morefrequent in online preparatory areas where students must collaborateactively to be successful. Cognitive presence levels also increasein such activities. Fig. 3 documents that when the activity turns backto the standard threaded discussion format, we see a relative declinein indicators of learner self- and co-regulation as well as a decline incognitive presence, though a slight increase in social presence.

As can be seen in Figs. 4 and 5 there is a correlation between learn-ing presence and social presence. A metric of presence density wascalculated by dividing instances of the various forms of presence bythe total numbers of posts in specific learning activities. Fig. 4 indi-cates that learning presence density per small group is correlatedwith social presence density within the collaborative activities.Fig. 5 shows a similar pattern when the message rather than thegroup is used as the unit of analysis. The average frequency of learn-ing presence per message is correlated with the average frequency ofsocial presence per message in collaborative activities.

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

Module 1 Module 2 Module 3 Module 4 Module 5

Average Presence by Studentin Discussions by Module

Course B

TP SP CP LP

Fig. 2. Instances of teaching presence (TP), social presence (SP), cognitive presence(CP), and learning presence (LP) in course B.

0

5

10

15

20

25

30

35

1 2 3 4

SP by Collaborative Activity LP by Collaborative Activity

Fig. 4. Learning presence density per small collaborative group.

92 P. Shea et al. / Internet and Higher Education 15 (2012) 89–95

3.4. Analysis of learning presence and course grades

The strength of the linear relationships among grades, learningpresence, teaching presence, social presence, and cognitive presenceswas examined by the means of Pearson r and partial correlations.Grades for the course were based on a variety of learning activitiesincluding module discussions; debate group participation, written as-signments, a research paper, and case study analyses. Table 1 displaysthe bi-variate zero-order correlations among the study variables,presented below the diagonal line. As can be seen, the four constructs(LP, TP, SP, and CP) are moderately to strongly correlated. The largestcorrelations are between cognitive presence and teaching presence(r=.86, pb .001) and cognitive presence and social presence(r=.87, pb .001). Learning presence, on the other hand, shows thestrongest correlation with final course grades (r=.62, pb .001).

To estimate the relative contribution of each of the four constructsfrom the CoI model in predicting course grades, we computed thepartial correlations between final course grades and each construct,controlling for the rest of the constructs. The results are presentedon the first line of Table 1. Learning presence and final course gradesremained moderately related (r=.49, pb .001), after the variance at-tributed to social presence, teaching presence and cognitive presencewas accounted for. The relationship between teaching presence andfinal course grades remained positive as well, but not statistically

0

2

4

6

8

10

12

14

16

All Prep Areas Debate Discussion

Average Student PresenceCombined Debate Prep and Debate Discussion

Course B

SP CP LP

Fig. 3. Forms of presence in debate preparation areas and debate discussion in course B.

significant (r=.246, pN .05) after controlling for learning presence,social presence and cognitive presence. Social presence and cognitivepresence were virtually unrelated to final course grades when the ef-fect of the remaining three constructs in each analysis were partialedout.

In sum, the analyses suggest that relative to teaching presence,cognitive presence and social presence, learning presence is a betterpredictor of course grades, explaining 24% of the variance in thelatter. Controlling for learning presence, teaching presence and socialpresence in fact eliminates the observed positive relationshipbetween cognitive presence and grades. The same holds true for therelationship between social presence and grades when learningpresence, cognitive presence and teaching presence are considered.

4. Discussion and implications

Decades of research indicate that learner self-regulation is an im-portant predictor of learning (Zimmerman, 2008), yet self-regulatedlearning in online environments in a topic that is not well understood.This gap is important to address given the near universal recognitionthat online learning requires significant self-direction and regulation.

The CoI model represents a powerful framework for understand-ing online learning in collaborative pedagogical environments.

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

1 2 3 4

Average LP per message Average SP per message

Fig. 5. Average frequency of learning presence and social presence per message in col-laborative online activity.

Table 1Zero-order and partial correlations among the study variables.

Variables Coursegrades

Learningpresence

Teachingpresence

Socialpresence

Cognitivepresence

Coursegrades

– .49a .246 −.138 −.037

Learningpresence

.621a –

Teachingpresence

.476a .528a −

Socialpresence

.469a .762a .771a –

Cognitivepresence

.488a .697a .863a .870a –

a The correlation is significant at the .01 level (2-tailed); the partial correlations arehighlighted on line one.

CognitivePresence

TeachingPresence

SocialPresence

LearningPresence

Suggestion for a Revised CoI Model

Fig. 6. Revised Community of Inquiry model including “Learning Presence”.

93P. Shea et al. / Internet and Higher Education 15 (2012) 89–95

While it represents an ideal in which teachers and learners performthe same roles (expressed as teaching presence), it ignores some ofthe real world dynamics that shape and constrain much of onlinelearning in practice. Learners and instructors do not perform identicalroles and thus must engage in different behaviors to succeed. For ex-ample, in this study, forethought and planning, monitoring, and strat-egy use exhibited by students are quite distinctive from the conceptsof instructional design, facilitation of discourse, and direct instructionthat characterize teaching presence. Because learners are accountablefor their learning they exhibit distinctive behaviors, motivations, andstrategies that are more or less adaptive and effective — we suggestthat these are indicative of a distinct learning presence in onlineenvironments.

Specifying components of online learner self- and co-regulation isnecessary. We began this task through an examination of discoursethat occurred in two fully online courses. This student–student dis-course had previously been coded (unsuccessfully) using codingschemes that prior researchers have developed for the CoI frame-work. We identified concepts, examples, and patterns of learningpresence within hundreds of instances of online discourse. Webelieve that the categories and examples presented through thisnew coding scheme represent an initial and beneficial extension tothe CoI model.

Learning presence is evident where learners are asked to activelycollaborate. Far less forethought/planning, monitoring of learningand strategy use appear in whole class discussion than in collabora-tive activities. Asking students to collaborate more deeply, throughinstructional design that includes complex forms of collaboration ap-pears to foster learning presence. Teaching presence is thus the routeto better learning presence but relationship between forms of pres-ence is probably reciprocal given the correlations identified here.We suggest here that it is the duty of the instructor to gain awarenessof and make efforts to encourage learners to understand the benefitsof self-regulatory learning processes. Fig. 6 indicates a revised CoImodel that indicates what we hypothesize to be the relationshipsbetween the forms of presence based on work included here.

The lack of a correlation between course grades and measure ofcognitive presence suggests that the instructor did not value this con-struct as highly as other grades on class activities and further suggestsan ongoing divide between theory and practice. While course gradesare a narrow measure of learning in collaborative environments theydo remain an important indicator for students, faculty, andinstitutions. That learning presence is the only construct that was sig-nificantly correlated with course grades certainly suggests its impor-tance. We analyzed instances of student discourse and found thatforms of forethought and planning, monitoring, and strategy usethat typify the activities of successful classroom-based self-regulators also appear to support the success of online students asmeasured by course grades. Again we emphasize that the notion oflearning presence is not meant to diminish the shared instructional

roles characteristic of progressive collaborative forms of learning,but that further specifying the roles of learners in online environ-ments is beneficial.

Akyol and Garrison (2011) recently suggested that the CoI modelcan be seen to account for important dimensions that focus onlearners per se, for example meta-cognition. We agree with thisline of analysis but have some reservations with the conceptualframing. For example, while arguing for an examination of metacog-nition the authors state, “The importance of understanding metacog-nition in text-based online learning contexts becomes apparentwhen considering the increased responsibility for self-regulation(Akyol & Garrison, 2011, p. 3).” The argument we have made hereand in the past (e.g. Shea & Bidjerano, 2010) recognizes the impor-tance of learner self-regulation. Akyol and Garrison go on to cite(among others) Pintrich (Pintrich, Wolters, & Baxter, 2000) as a con-ceptual resource for understanding the significance of meta-cognition. We would agree that Pintrich does supply a good ground-ing for understanding what effective learners do when engaged inonline academic study. However, Pintrich's own lasting legacy isself-regulated learning (see e.g. Schunk, 2005), and he consideredmeta-cognition to be one component of the larger construct of learn-er self-regulation. This is demonstrated by the Motivated Strategiesfor Learning Questionnaire (MSLQ) (Pintrich, Smith, Garcia, &McKeachie, 1993), an instrument that measures self-regulated learn-ing in which Pintrich and his colleagues include metacognition asone of several dimensions of self-regulated learning. In otherwords, for Pintrich and other researchers, metacognition is a subsetof learner self-regulation and we would suggest that self-regulatedlearning is the larger and more inclusive conceptual lens throughwhich to investigate the roles of online learners as learners. Again,this is not to argue that teachers and learners in collaborative onlineenvironments do not share roles, including shared responsibilitiesfor teaching presence. It does suggest that we need to know moreabout effective online learners qua learners and that the literatureon learner self-regulation seems the richer source to support anddevelop such knowledge.

Kuhn (1977) argued that determining the superiority of one theo-ry over another was a matter of weighing competing values includingaccuracy, consistency, scope, and fruitfulness among others. In arelated sense Greeno (2006) argued that theoretical progress can bemade in a number of ways, ranging from improvements that add tothe scope of phenomena that a theory explains to improvementsthat increase the accuracy with which the theory accounts for phe-nomena it already explains. In this paper, we examine the degree towhich the CoI model represents an advantageous theoretical frame-work for understanding online learning based on some of these cri-teria. We suggest that the addition of learning presence to the CoImodel represents progress by these standards. Again, the roles of

M5 — Appraising levelof interestand engagement

Comments betweenonline learners about selfor others' engagement,interest, commitmentor participation

“I like being on the ‘con’end of this discussion. Iam not a supporter ofoutsourcing.”(Azevedo et al., 2004a,

2004b)

Appendix A (continued)

94 P. Shea et al. / Internet and Higher Education 15 (2012) 89–95

teachers and learners may be similar in collaborative environments,but they are not identical. Further explicating the roles of onlinelearners by drawing on theories of learner self-regulation providesadded scope, accuracy and we believe will make the model increas-ingly fruitful in describing and explaining online learning.

M6 — Noting one's ownor group's learningbehavior

Statements aboutindividual or group'sstrengths/weaknesses(metacognitiveknowledge) or changesin thinking betweenonline learners.

“I am more of a hands-onlearner.”

(Emergent)

Strategy useS1 — Seeking, offering,providing helpor information

Online learnersrequesting, offering, orproviding assistance orinformation related tolearning materials,activities, tasks or goals.

“If you need anyassistance, please let meknow what I can do tohelp you out.”(Curtis & Lawson, 2001)

S2 — Seeking, offering,providing clarification

Seeking, offering,providing clarificationbetween online learners

“Just as a point ofclarification, are youseeking a critique of thespecific informationcontained in the readingsor are you concerned withour opinions about howthe material is presented?”

(Emergent)

5. Limitations and future research

This studywas limited to archived courses and, as a result, wewerenot able to interview students to probemore deeply into the nature oftheir efforts at collaboration as it related to forethought/planning,monitoring and strategy use. At the same time, the development ofthe learning presence coding scheme was based on just two types ofonline learning activities, full class discussions and small groupprep areaswhere students collaborated towrite a position paper. Con-tent analysis of other types of learning activities, such as student-authored reflective essays or learning journals may yield evidence ofother behaviors that broaden our understanding of learning presenceas it relates to individual/private and group/public learning. Other av-enues for further exploration include understanding the specific rolesand contributions of students with high levels of learning presenceto group collaborative work which is product-focused and sharedknowledge construction which is process-oriented.

Code and source Description Example

Forethought and planningFP1 — Goal setting Online learner discourse

establishing desiredtangible and intangibleoutcomes

“At the end of next week,as a team, we have tosubmit a summary of ourdiscussion points.”

(Zimmerman, 1989)

FP2 — Planning Online learnersconsidering approaches,procedures, or tasks to beused to attain goals.

“Why don’t we list (allof us) what we perceiveto be the cons ofoutsourcing.”

(Zimmerman, 1989)

FP3 — Coordinating &assigning tasks toself and others

Online learnersdistributing, sequencingtasks and sub-tasks toothers/self for futurecompletion

“Are you picking this[task] up next?”

(Emergent)

MonitoringM1 — Checking forunderstanding

Online learners seekingverification ofunderstanding of tasks,events or concepts fromother online learners.

“Are we sure thateverything has beencited correctly?”(Emergent)

M2 — Identifyingproblems or issues

Online learners drawingattention of other onlinelearners to difficulties thatmay interfere withcompletion of tasks orother outcomes

“I am unable to open thequiz. Does anyone elsehave this problem?”(Emergent)

M3 — Noting completionof tasks

Comments betweenonline learners thatindicate that certain tasksor activities have beenfinished to supportattaining a goal.

“I did some researchand then typed up theemployer section.”(Azevedo et al., 2004a,

2004b)

M4 — Evaluating thequality of an endproduct, its content orits constituent parts

Statements betweenonline learners thatjudge the accuracy,comprehensiveness,relevance or other aspectsof an end product or itscomponents.

“I fully agree with thisconcept. This is definitelyan area we should buildupon. “

(Azevedo et al., 2004a,2004b)

S3 — Advocating effort Encouraging or urgingother online learners tocontribute to the onlinegroup

“Has everyonecontributed their pieces?”(Curtis & Lawson, 2001)

Appendix A. Learning presence coding scheme

Table B1Course debates inter-rater reliability.

Appendix B. Inter-rater reliability

Instructor A Instructor B

kappa CR kappa CR

Pre Post Pre Post Pre Post Pre Post

Con 1 0.3251 0.6436 0.9823 0.9938 0.3538 0.9025 0.6346 0.9649Con 2 0.6398 0.8229 0.9552 0.9785 0.4654 0.9407 0.7368 0.9781Pro 1 0.2479 1.0000 0.6000 1.0000 0.8140 1.0000 0.9307 1.0000Pro 2 0.6172 1.0000 1.0000 0.8302 0.5462 1.0000 0.7816 1.0000Discussion 0.5988 1.0000 0.8366 1.0000 0.4470 0.7500 0.9332 0.9833

Table B2Course discussions inter-rater reliability.

Instructor A Instructor B

Kappa CR Kappa CR

Pre Post Pre Post Pre Post Pre Post

M1 0.6479 0.9283 0.9574 0.9894 0.7460 1.0000 0.9203 1.0000M2 0.2796 1.0000 0.8366 1.0000 0.3086 0.9025 0.8867 0.9649M3 0.2255 1.0000 0.9468 1.0000 0.5405 0.9309 0.8923 0.9848M4 0.2168 0.7927 0.9175 0.9801 0.4908 1.0000 0.9759 1.0000M5 0.5359 1.0000 0.9286 1.0000 0.3767 1.0000 0.7270 1.0000

References

Akyol, Z., & Garrison, D. R. (2008). The Development of a Community of Inquiry overTime in an Online Course: Understanding the Progression and Integration ofSocial, Cognitive and Teaching Presence. Journal of Asynchronous Learning Network,12(3–4), 3–22.

Akyol, Z., & Garrison, D. R. (2011). Assessing metacognition in an online community ofinquiry. The Internet and Higher Education, 14(3), 183–190. doi:10.1016/j.iheduc.2011.01.005

95P. Shea et al. / Internet and Higher Education 15 (2012) 89–95

Allen, M., Bourhis, J., Burrell, N., & Mabry, E. (2002). Comparing student satisfactionwith distance education to traditional classrooms in higher education: A meta-analysis. American Journal of Distance Education, 16, 83–97. doi: http://dx.doi.org/10.1207/S15389286AJDE1602_3

Allen, I. E., & Seaman, J. (2008). Staying the Course: Online Education in the United States,2008. Needham MA: Sloan Consortium. Retrieved from http://sloanconsortium.org/publications/survey/staying_course

Allen, E., & Seaman, J. (2010). Class differences: Online education in the United States,2010. Needham: MA: Sloan Consortium. Retrieved from http://sloanconsortium.org/publications/survey/pdf/class_differences.pdf

Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching pres-ence in a computer conferencing environment. Journal of Asynchronous LearningNetworks, 5, 1–17.

Azevedo, R., Cromley, J. G., & Seibert, D. (2004a). Does adaptive scaffolding facilitatestudents' ability to regulate their learning with hypermedia? Contemporary Educa-tional Psychology, 29, 344–370.

Azevedo, R., Guthrie, J. T., & Seibert, D. (2004b). The role of self-regulated learning infostering students' conceptual understanding of complex systems with hyperme-dia. Journal of Educational Computing Research, 30(1), 87–111. doi:10.2190/DVWX-GM1T-6THQ-5WC7

Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., Wallet, P. A.,Fiset, M., & Huang, B. (2004). How does distance education compare to classroominstruction? A meta-analysis of the empirical literature. Review of Educational Re-search, 74, 379–439. doi: http://dx.doi.org/10.3102/00346543074003379

Bixler, B. A. (2008). The effects of scaffolding student's problem-solving processvia question prompts on problem solving and intrinsic motivation in an onlinelearning environment. Unpublished PhD dissertation. The Pennsylvania State Uni-versity, State College, Penn

Bliss, C. A., & Lawrence, B. (2009). From posts to patterns: a metric to characterize dis-cussion board activity in online classes. Journal of Asynchronous Learning Networks,13(2), 15–32.

Chang, M. M. (2007). Enhancing web-based language learning through self-monitoring. Journal of Computer Assisted Learning, 23, 187–196. doi: http://dx.doi.org/10.1111/j.1365-2729.2006.00203.x

Chung, S., Chung, M. -J., & Severance, C. (1999). Design of support tools and knowledgebuilding in a virtual university course: Effect of reflection and self-explanationprompts. Paper presented at the WebNet 99 World Conference on the WWW andInternet Proceedings, Honolulu, Hawaii. (ERIC Document Reproduction ServiceNo. ED448706).

Coll, C., Engel, A., & Bustos, A. (2009). Distributed teaching presence and participants’activity profiles: a theoretical approach to the structural analysis of asynchronouslearning networks. European Journal of Education, 44(4), 521–538. doi:10.1111/j.1465-3435.2009.01406.x

Cook, D. A., Dupras, D. M., Thompson, W. G., & Pankratz, V. S. (2005). Web-based learn-ing in residents' continuity clinics: A randomized, controlled trial. Academic Medi-cine, 80, 90–97. doi: http://dx.doi.org/10.1097/00001888-200501000-00022

Crippen, K. J., & Earl, B. L. (2007). The impact of Web-based worked examples and self-explanation on performance, problem solving, and self-efficacy. Computers in Edu-cation, 49, 809–821. doi: http://dx.doi.org/10.1016/j.compedu.2005.11.018

Curtis, D. D., & Lawson, M. J. (2001). Exploring collaborative online learning. Journal ofAsynchronous Learning Networks, 21(1), 21–34.

Dewey, J. (1933). How we think. A restatement of the relation of reflective thinking to theeducative process. Boston, MA: D. C. Heath.

Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problemsof two paradoxes. Journal of Clinical Epidemiology, 43(6), 543–549. doi: http://dx.doi.org/10.1016/0895-4356(90)90158-L

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based envi-ronment: Computer conferencing in higher education. The Internet and Higher Ed-ucation, 2, 87–105. doi: http://dx.doi.org/10.1016/S1096-7516(00)00016-6

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence,and computer conferencing in distance education. American Journal of DistanceEducation, 15(1), 7–23. doi:10.1080/08923640109527071

Greeno, J. (2006). Theoretical and practical advances through research on learning. In J.Green, G. Camilli, & P. Elmore (Eds.), Handbook of complementary methods on edu-cation research (pp. 795–822). Mahwah, NJ: Lawrence Erlbaum Associates.

Kuhn, T. (1977). The essential tension: Selected studies in scientific tradition and change.Chicago: University of Chicago Press.

Lipmann, M. (2003). Thinking in education. New York: Cambridge University Press.Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis inmass communi-

cation: Assessment and reporting of inter-coder reliability.Human Communication Re-search, 28, 587–604. doi: http://dx.doi.org/10.1111/j.1468-2958.2002.tb00826.x

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learningstudies. Washington, D.C.: U.S. Department of Education, Office of Planning, Evalua-tion, and Policy Development. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

Nelson, B. C. (2007). Exploring the use of individualized, reflective guidance in an ed-ucational multi-user virtual environment. Journal of Science Education and Technol-ogy, 16, 83–97. doi: http://dx.doi.org/10.1007/s10956-006-9039-x

Parsad, B., & Lewis, L. (2008). Distance education at degree-granting postsecondary in-stitutions: 2006–07 (NCES 2009-044). National Center for Education Statistics, Insti-tute of Education Sciences. Washington, DC: U.S. Department of Education.Retrieved from http://www.eric.ed.gov/PDFS/ED503770.pdf

Pawan, F., Paulus, T., Yalcin, S., & Chang, C. (2003). Online learning: patterns of engage-ment and interaction among in-service teachers. Language Learning & Technology,7(3), 119–140.

Pierce, C. S. (1955). The fixation of belief. In J. Buchler (Ed.), Philosophical writings ofPierce (pp. 5–22). New York: Dover.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and pre-dictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Ed-ucational and Psychological Measurement, 53, 801–813. doi:http://dx.doi.org/10.1177/0013164493053003024

Pintrich, P. R., Wolters, C., & Baxter, G. (2000). Assessing metacognition and self-regu-lated learning. In G. Schraw & J. Impara (Eds.), Issues in the Measurement of Meta-cognition (pp. 43–97). Lincoln, NE: Buros Institute of Mental Measurements.

Steadman, Rex, & Graciano (2006). Researching the complexity of classroom interac-tion. In J. Green, G. Camili, & P. Elmore (Eds.), Handbook of complementary methodsin educational research. Mahwah, NJ: Lawrence Erlbaum.

Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (1999). Assessing social presencein asynchronous text-based computer conferencing. Journal of Distance Education,14, 51–70.

Saito, H., & Miwa, K. (2007). Construction of a learning environment supportinglearners' reflection: A case of information seeking on the Web. Computers in Educa-tion, 49, 214–229. doi: http://dx.doi.org/10.1016/j.compedu.2005.07.001

Schunk, D. H. (2005). Self-regulated learning: The educational legacy of Paul R. Pintrich.Educational Psychologist, 40, 85–94. doi: http://dx.doi.org/10.1207/s15326985ep4002_3

Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy,self-regulation, and the development of a communities of inquiry in online andblended learning environments. Computers in Education, 55, 1721–1731. doi:http://dx.doi.org/10.1016/j.compedu.2010.07.017

Shea, P., Li, C., Swan, K., & Pickett, A. (2005). Developing learning community in onlineasynchronous college courses: The role of teaching presence. Journal of Asynchro-nous Learning Networks, 9, 59–82.

Shea, P., Hayes, S., & Vickers, J. (2010). Online instructional effort measured throughthe lens of teaching presence in the Community of Inquiry framework: A Re-examination of measures and approach. International Review of Research in Openand Distance Learning, 11, 127–154.

Shea, P., Hayes, S., Vickers, J., Uzuner, S., Gozza-Cohen, M., Mehta, R., & Valtcheva, A.(2010). A re-examination of the Community of Inquiry framework: Social networkand quantitative content analysis. The Internet and Higher Education, 13, 10–21. doi:http://dx.doi.org/10.1016/j.iheduc.2009.11.002

Shen, P. D., Lee, T. H., & Tsai, C. W. (2007). Applying Web-enabled problem-basedlearning and self-regulated learning to enhance computing skills of Taiwan's voca-tional students: A quasi-experimental study of a short-term module. ElectronicJournal of e-Learning, 5, 147–156.

Sim, J., & Wright, C. C. (2005). The kappa statistic in reliability studies: use, interpreta-tion, and sample size requirements. Physical Therapy, 85, 257–268.

Staklis, S. (2010). Profile of undergraduate students 2007–2008. U.S. Department of Ed-ucation, National Center for Education Statistics. Washington, DC: Institute of Educa-tion Sciences. Retrieved from http://nces.ed.gov/pubs2010/2010205.pdf.

Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M.,et al. (2006). Teaching courses online: A review of the research. Review of Educa-tional Research, 76, 93–135. Doi: http://dx.doi.org/10.3102/00346543076001093

Wang, K. H., Wang, T. H., Wang, W. L., & Huang, S. C. (2006). Learning styles and forma-tive assessment strategy: Enhancing student achievement in Web-based learning.Journal of Computer Assisted Learning, 22, 207–217 http://dx.doi.org/10.1111/j.1365-2729.2006.00166.x

Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005).Whatmakes the difference? A practicalanalysis of research on the effectiveness of distance education. Teachers College Re-cord, 107, 1836–1884. doi: http://dx.doi.org/10.1111/j.1467-9620.2005.00544.x

Zimmerman, B. J. (1986). Becoming a self-regulated learner: What are the key sub-processes? Contemporary Educational Psychology, 11, 307–313. doi: http://dx.doi.org/10.1016/0361-476X(86)90027-5

Zimmerman, B. J. (1989). A Social Cognitive View of Self-Regulated Academic Learning.Journal of Educational Psychology, 81(3), 329–339.

Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M.Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation(pp. 13–39). San Diego, CA: Academic Press.

Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical back-ground, methodological developments, and future prospects. American EducationalResearch Journal, 45, 166–183. doi: http://dx.doi.org/10.3102/0002831207312909