The Effect of Online Collaborative Learning on Middle School

THE EFFECT OF ONLINE COLLABORATIVE LEARNING ON MIDDLE

SCHOOL STUDENT SCIENCE LITERACYAND SENSE OF COMMUNITY

by

Jillian Leigh Wendt

Liberty University

A Dissertation Presented in Partial Fulfillment

of the Requirements for the Degree

Doctor of Education

Liberty University

April 2013

2

The Effect of Online Collaborative Learning on Middle School Student Science

Literacy and Sense of Community

by Jillian Leigh Wendt

A Dissertation Presented in Partial Fulfillment

of the Requirements for the Degree

Doctor of Education

Liberty University, Lynchburg, VA

April 2013

APPROVED BY:

Amanda J. Rockinson-Szapkiw, EdD, Committee Chair

Kurt Y. Michael, PhD, Committee Member

Jennifer L. Courduff, PhD, Committee Member

3

ABSTRACT

This study examines the effects of online collaborative learning on middle school

students’ science literacy and sense of community. A quantitative, quasi-experimental

pretest/posttest control group design was used. Following IRB approval and district

superintendent approval, students at a public middle school in central Virginia completed

a pretest consisting of the Misconceptions-Oriented Standards-Based Assessment

Resources for Teachers (MOSART) Physical Science assessment and the Classroom

Community Scale. Students in the control group received in-class assignments that were

completed collaboratively in a face-to-face manner. Students in the experimental group

received in-class assignments that were completed online collaboratively through the

Edmodo educational platform. Both groups were members of intact, traditional face-to-

face classrooms. The students were then post tested. Results pertaining to the MOSART

assessment were statistically analyzed through ANCOVA analysis while results

pertaining to the Classroom Community Scale were analyzed through MANOVA

analysis. Results are reported and suggestions for future research are provided.

Keywords: middle school, misconceptions, science, collaborative learning,

technology, science literacy, sense of community

4

Table of Contents

CHAPTER ONE: INTRODUCTION 10

Introduction ........................................................................................................10

Background ........................................................................................................13

Problem Statement .............................................................................................19

Purpose Statement ..............................................................................................20

Significance of the Study ...................................................................................22

Research Question(s)..........................................................................................24

Hypothesis or Hypotheses ..................................................................................24

Identification of Variables ..................................................................................26

Definitions ..........................................................................................................28

CHAPTER TWO: REVIEW OF THE LITERATURE 31

Introduction……………………………………………………………………31

Social Development Theory…………………………………………………..34

Collaborative Learning………………………………………………………..38

Collaborative Learning in the Science Classroom………………………….40

Computer-Mediated and Web-based Collaborative Learning……………...42

Community and Effective Online Learning…………………………………...46

Communities of Practice Theory……………………………………………...47

Sense of Community…………………………………………………………..51

Review of Literature…………………………………………………………..55

Science Literacy……………………………………………………………….58

Significance……………………………………………………………………64

CHAPTER THREE: METHODOLOGY 67

Introduction ........................................................................................................67

Design.................................................................................................................67

Questions and Hypotheses .................................................................................68

Participants .........................................................................................................70

Setting.................................................................................................................72

Classroom Teachers…………………………………………………………72

5

Overview of School…………………………………………………………72

Science Classroom………………………………………………………….73

Collaboration Settings………………………………………………………74

Explanation of Collaborative Activities…………………………………….77

Instrumentation...................................................................................................82

Procedures ..........................................................................................................88

Data Analysis .....................................................................................................90

Research Question One……………………………………………………..90

Research Question Two…………………………………………………….92

CHAPTER FOUR: FINDINGS 97

Restatement of Purpose………………………………………………………..97

Research Questions and Hypotheses………………………………………….97

Demographics………………………………………………………………....99

Research Question One………………………………………………………100

Assumption Testing……………………………………………………….100

Normality………………………………………………………………….101

Linearity…………………………………………………………………...104

Variance…………………………………………………………………...104

Homogeneity of Regression Slopes………………………………………104

Reliability of Covariates………………………………………………….105

Results…………………………………………………………………….105

Hypothesis Testing………………………………………………………..…105

Descriptive Statistics……………………………………………………...105

Analysis…………………………………………………………………..106

Results of Hypothesis One………………………………………………..107

Research Question Two……………………………………………………...108

Assumption Testing………………………………………………………109

Normality…………………………………………………………………109

Linearity…………………………………………………………………..111

Homogeneity of Variance and Covariance……………………………….112

Results…………………………………………………………………….113

6

Hypothesis Testing…………………………………………………………114

Descriptive Statistics…………………………………………………….114

Analysis………………………………………………………………….115

Results of Hypothesis Two………………………………………………116

Hypothesis Three…………………………………………………………116

Results of Hypothesis Three……………………………………………...117

Hypothesis Four…………………………………………………………..117

Results of Hypothesis Four……………………………………………….118

Summary…………………………………………………………………..…118

Additional Analysis………………………………………………………….120

CHAPTER FIVE: DISCUSSION 122

Introduction…………………………………………………………………..122

Statement of Problem………………………………………………………..122

Review of Methodology……………………………………………………..123

Summary of Results………………………………………………………….124

Research Question One…………………………………………………...124

Research Question Two…………………………………………………..125

Additional Analysis………………………………………………………126

Discussion of Results………………………………………………………..126

Research Question One………………………………………………..…126

Research Question Two…………………………………………………..129

Implications…………………………………………………………………..131

Theoretical Implications………………………………………………….131

Limitations…………………………………………………………………...133

Implications for Practice and Methodological Implications…………………136

Implications for Future Research…………………………………………….137

Conclusion…………………………………………………………………...139

REFERENCES 141

APPENDICES 163

7

List of Tables

1 Participant Demographics……………………………………………………………71

2 Description of Sequence, Content, and Alignment of Activities…………………….78

3 National Science Education Standards……………………………………………….79

4 Virginia Standards of Learning for Physical Science…………………………………80

5 MOSART Alignment………………………………………………………………….82

6 Description of MOSART Instrument………………………………………………….88

7 Organization of Statistical Analysis of Data…………………………………………..95

8 Assumption Testing for Research Question One……………………………………..105

9 Descriptive Statistics for MOSART Pre-Test Data…………………………………..106

10 Descriptive Statistics for MOSART Post-Test Data………………………………...106

11 Assumption Testing for Research Question Two…………………………………...113

12 Descriptive Statistics for CCS Community Subscale Pre-Test……………………...114

13 Descriptive Statistics for CCS Overall Community Post-Test……………………...115

14 Results of Statistical Analysis per Hypothesis………………………………………119

15 Descriptive Statistics for Additional Analysis………………………………………121

16 Organization of Theoretical Framework, Questions, Design, and Data…………….133

8

List of Figures

1 Edmodo Screenshot……………………………………………………………………75

2 Arrangement of Classrooms………………………………………………………...…77

3 Scatterplot of MOSART Data………………………………………………………...101

4 Histograms for Normality Testing of Research Question One……………………….103

5 Histograms for Normality Testing of Research Question Two………………………110

6 Scatterplot of Classroom Community Scale Data……………………………………112

9

List of Abbreviations

Classroom Community Scale (CCS)

Communities of Inquiry (CoI)

Communities of Practice (CoP)

Misconceptions-Oriented Standards-Based Resources for Teachers (MOSART)

10

CHAPTER ONE: INTRODUCTION

Introduction

Science literacy is a set of skills that enables an individual to understand, critique,

and apply scientific knowledge and processes (Impey, Buxner, Antonellis, Johnson, &

King, 2011) for the purposes of personal enjoyment, productivity, and participation in

current affairs (American Association for the Advancement of Science, 1993). As such,

science literacy has been identified as imperative to future civic and scientific success,

especially in the face of growing technological and scientific advances and global

ecological crises (AAAS, 1993; Impey et al., 2011; Miller, 2007). The topic of science

literacy is important as science continuously shapes human culture (AAAS, 1993; Impey

et al, 2011). Unfortunately, national and international studies have shown that students

are lacking in science literacy (AAAS, 1991; National Science Foundation & National

Center for Science and Engineering Statistics, 2010; OECD, 2007). Of increasing

concern in the United States is the lack of science literacy among adolescents despite

multiple revisions to state and national science standards and related teaching initiatives

(AAAS, 1993; Organisation for Economic Co-operation and Development, 2007).

A key component to science literacy is the understanding of science concepts and

the level of mastery of conceptual science knowledge (Laugksch, 2000; Roberts, 2007).

This is reflected in the evaluative sense of the term literacy, which implies the level of

mastery of a particular topic (Laugksch, 2000). The acquisition of knowledge and

mastery of scientific concepts may only be attained when scientific misconceptions are

identified and dispelled (AAAS, 2013; Burgoon, Heddle, & Duran, 2011; Harvard,

2011). Misconceptions may be defined as science conceptions that are held by students

that are different from scientific knowledge that is currently accepted within the science

11

community at large (Burgoon et al., 2011), which may be preconceived or a result of

misinformed teaching (AAAS, 1993; Harvard, 2011). Misconceptions, thus, have been

identified as an important aspect of overall science literacy (AAAS, 1993).

Revisions of standards and related teaching methods have been directed at

increasing science literacy and, more specifically, decreasing overall science

misconceptions (AAAS, 1990; AAAS, 1993; National Academy of Sciences, 1996). In

the past, science was taught in a didactic lecture-based format (Lunetta, Hofstein, &

Clough, 2007). This traditional approach was based upon behavioral theories of learning

(Skinner, 1957) with the underlying assumption that objective knowledge should be

transmitted to individual students for absorption and recall. This method of teaching has

been found ineffective and ill suited to promote science achievement and advance science

literacy (Lunetta et al., 2007; Scott, Asoko, & Leach, 2007). Science instruction that

utilizes the constructivist approach and authentic hands-on learning strategies are better

suited for promoting science achievement and advance literacy (Fang & Wei, 2010; Guo,

2007). Numerous research studies have shown that collaborative activities in the

traditional science classroom result in more effective science learning (Bell, Urhahne,

Schanze, & Ploetzner, 2010; Mäkitalo-Siegl, Kohnle, & Fischer, 2011; Raes, Schellens,

Wever, & Vanderhoven, 2012). Now, as technology advances and online education

exponentially grows, there is a need to examine online teaching methods and tools that

effectively support aspects of learning and, more specifically, aspects of science literacy

(Bell et al., 2010). Researchers need to examine if collaborative teaching methods that

support learning and science literacy in the traditional face-to-face classroom can be

12

mimicked, or even enhanced, by the use of web-based tools and online instruction

(Bergtrom, 2011; Lunetta et al., 2007; Underwood, Smith, Luckin, & Fitzpatrick, 2008).

As misconceptions regarding scientific concepts have been identified as a component of

science literacy, misconceptions were examined in this study.

Sense of community has also been recognized as important and foundational in

collaborative learning both inside and outside of the classroom as well as in traditional

and online classrooms (Abfalter, Zaglia, & Mueller, 2012; Dawson, 2008; Rovai, 2002);

therefore, sense of community was examined as well. Sense of community for the

purpose of this study was defined as how members feel that they belong in a group and

how their needs are met within the group, which involves the creation of a social

community (Abfalter et al., 2012; Wenger, 1998, Wenger, White, & Smith, 2009). Sense

of community is important as it has been shown to be associated with student motivation

and an increase in student science knowledge (Lunetta et al., 2007). Specifically, a

learning environment that fosters community through peer communication and

collaboration is necessary to mimic how the scientific community functions in the real

world (Lunetta et al., 2007). Since recent learning theory, such as connectivism,

proposes that learning is based on social connections fostered through collaboration and

centered on communities, there exists a need for examination of the relationship between

sense of community and its support of science literacy (Siemens, 2006).

It is widely accepted that science knowledge is best attained through

constructivist activities, especially those that take on a social constructivist approach and

encourage peer dialogue, inquiry, and reflection (Scott et al., 2007). Furthermore,

communities of practice theory maintains that involvement in a learning community

13

increases student learning, connectedness, and sense of community (Wenger, 1998).

According to distance learning theory, the outcomes of traditional teaching methods and

online teaching methods are equivalent (Clark & Mayer, 2011); thus an examination of

the effects of face-to-face collaborative learning and online collaborative learning would

yield evidence to either support or refute this claim, which would add to the current body

of knowledge to further influence current teaching pedagogy.

This chapter will provide an extensive background of the importance of science

literacy as it relates to understanding of scientific concepts, sense of community, and

collaborative learning. The current literature is discussed, as will the gap in the literature

that will lead to the problem statement, purpose statement, and the significance of the

study. The research questions and hypotheses are stated, variables are identified, and

terms are defined.

Background

Science literacy has been a topic of concern for educators in the field of science

since the early 1950s (Laugksch, 2000). Science literacy was originally seen as

necessary only for those who pursued higher education or a career in the science field

(Liu, 2009). During the 1970s, science literacy was deemed important for all students in

order to be productive and capable citizens in a rapidly advancing world (Liu, 2009). In

the 1980s, science literacy began to encompass not only science and literacy, but rather

mathematical and technological knowledge as well (Liu, 2009). In the 1990s, the concept

became even more complex when the National Research Council added the requirements

of understanding scientific processes (Liu, 2009). Finally, in the 2000s, with the

continuing increase in scientific and technological advances coupled with increased

14

competition with other nations, science literacy again moved to the forefront of core

educational planning (Virginia Department of Education, 2011b).

Numerous attempts over the course of the years have sought to determine what

science literacy really means with some experts citing the importance of science

knowledge while other experts placed the focus on literacy (American Association for the

Advancement of Science, 1993; Impey et al., 2011; Liu, 2009). Likewise, throughout the

research, some experts have focused on the acquisition of science proficiency (Impey et

al., 2011), others on scientific reading literacy (Liu, 2009), and still others on the use of

technology as it relates to science (AAAS, 2013; International Technology and

Engineering Educators Association, 2011; Virginia Department of Education, 2011b).

The AAAS’s (1993) Project 2061 defined science literacy as the knowledge and habits of

mind related to science, mathematics, and technology that individuals must acquire in

order to live interesting, responsible, and productive lives. The National Science

Foundation (2004) defined science literacy as the knowing of basic scientific facts and

concepts and having a general understanding of how science works. Despite the

numerous operational definitions of science literacy, researchers have come to the

consensus that society has been and remains lacking in scientific knowledge, scientific

skill, and application of scientific processes and methods to current events (Laugksch,

2000; Miller, 2007; Organisation for Economic Co-operation and Development, 2007).

Of particular concern is the lack of scientific knowledge, the ability to apply knowledge

to current events, and the increased misconceptions of scientific principles held by

adolescents (Harvard, 2011; Impey et al., 2011).

15

Recently, the Programme for International Student Assessment (PISA) 2006

survey found that on a scale of 1-6, with 1 being the lowest level of science literacy and 6

being the highest level of science literacy, the vast majority of students in ten countries

did not reach a science literacy equivalent to a level 2 (Organisation for Economic Co-

operation and Development, 2007). Over 400,000 students from 57 countries were

surveyed as participants in the PISA 2006 (OECD, 2007). Approximately 2% of students

in nine of the countries surveyed exhibited knowledge consistent with a Level 6, the

highest level of scientific literacy. Three key areas noted within the PISA 2006 survey

were identifying scientific issues, explaining phenomena using scientific processes, and

utilizing scientific evidence (Organisation for Economic Co-operation and Development,

2007).

The PISA 2006 indicated an alarming percentage of students performed at a low

proficiency level (level 2 and below) in the areas of science and technology, thus,

potentially limiting full and productive participation in life (Organisation for Economic

Co-operation and Development, 2007). While the majority of students (92%) believed

that science and technology improves living conditions, only 57% of students perceived

science and technology have personal relevance (Organisation for Economic Co-

operation and Development, 2007). Only 29.3% of students surveyed exhibited

proficiency with working within situations that required integration of science or

technology and making connections with real-life situations. With rapid changes in the

environment and advances in technology that allow cloning, stem cell research,

biological warfare, and the creation of genetically modified foods, it is imperative that

16

adolescents procure skills allowing informed scientific decision making (Impey et al.,

2011).

The acquisition of scientific knowledge that allows connections with daily living is

integrally tied to misconceptions of science. Misconceptions may manifest as

preconceptions, oftentimes obtained through past experiences, or as naïve explanations to

explain scientific phenomena (Burgoon et al., 2011). Significant barriers to attainment of

science knowledge may exist if misconceptions are not identified or reduced (American

Association for the Advancement of Science, 1993; Burgoon et al., 2011; Harvard, 2011).

In order to encourage participation in a rapidly advancing scientific world, there is a

growing need to examine methods to reduce misconceptions of science (Burgoon et al.,

2011), increase science achievement, and foster achievement of science literacy (Roberts,

2007).

Coupled with advances in science, advances in technology seem to be progressing

at ever-increasing speed. Technology is now at the forefront of education with

policymakers pushing the integration of instructional technology in the classroom and

online course requirements (VDOE, 2012b) as well as demanding the acquisition of

technological literacy skills (ITEAA, 2011; VDOE, 2011b) in order to ensure

competitiveness in the global economy (Williams, 2009). Within education specifically,

the availability of numerous tools, programs, and applications allows educators to

incorporate technology in instruction at almost any time and any place. The proliferation

of online tools has provided convenient access to learning materials for educators and

students alike and has led to a push towards collaborative learning (Lou, Abrami, &

17

d’Apollonia, 2001). Through the use of technological tools, collaboration has become

more convenient by transcending boundaries of time and location.

Collaborative learning allows individuals to work together for a common purpose

or towards a common goal. Research has consistently shown that collaboration provides

both academic and social benefits (Bye, Smith, & Rallis, 2009; Miller & Benz, 2008; Yu,

Tian, Vogel, & Kwok, 2010). Additionally, recent studies have shown that collaborative

learning through the use of technology has become an integral part of today’s classrooms

(Keser, Uzunboylu, & Ozdamli, 2011; Yang & Chang, 2012). Commonly referred to as

technology supported or computer-mediated collaborative learning, collaboration through

the use of technological tools allows learners many of the same benefits as traditional

collaborative activities in more effective and efficient ways (Miller & Benz, 2008).

Using the basic tenets of constructivism, learners are able to actively engage in learning

(Dewey, 1997) and participate in a cooperative learning community (Dewiyanti, Brand-

Gruwel, Jochems, & Broers, 2007; Donne, 2012; Vygotsky, 1986). More specifically,

learners may be able to construct scientific concepts through cooperative activities with

others in an online environment (Vygotsky, 1986).

While multiple studies have examined the effects of collaboration on student

achievement in general and found positive results (Bluic, Ellis, Goodyear, & Piggott,

2010; Winters & Alexander, 2011; Yang & Chang, 2012), researchers need to examine

the influence of collaboration on science achievement, science misconceptions, and

science literacy (Anderson, 2007a; Lunetta, et al., 2007). A study by Jeong and Chi

(2007) investigated students’ gains in common knowledge of scientific material after

engaging in face-to-face collaborative learning activities and found that knowledge

18

construction increased as well as misconceptions shared among peers, thus indicating that

some benefits to learning may exist as well as disadvantages that warrant closer attention.

More importantly, as the integration of technology increases in the classroom and

demands for online education expand, there exists a need to examine the differences

between face-to-face and computer-mediated collaborative learning (Tutty & Klein,

2008) in an effort to better understand how to structure online learning tasks in ways that

facilitate social interaction (Nuankhieo, Tsai, Goggins, & Laffey, 2007). One study

investigated whether the positive effects of student achievement attained through face-to-

face collaboration was also present in computer-mediated collaboration (Tutty & Klein,

2008). Results indicated that student performance through face-to-face collaboration

exceeded student performance using computer-mediated collaboration; however, student

interaction, discussion, and inquiry were more frequent among students participating in

computer-mediated collaboration (Tutty & Klein, 2008). Benefits were shown, therefore,

to exist for both face-to-face and computer-mediated collaboration. There is still a need

for further examine how to determine which collaborative structures are best-suited for

each mode of learning (Tutty & Klein, 2008). Therefore, it is timely that research be

conducted that seeks to understand how online collaboration may assist in knowledge

acquisition, application to real-world events, and identification of commonly held

misconceptions specifically in the area of science.

Additionally, sense of community has become an increasingly important concept as

collaborative activities lead to the creation of learning communities in the classroom

(Rovai, 2002b). This is supported by the literature on effective education rooted in

constructivism and social constructivism. Further, a point of consensus among many

19

researchers studying effective online education is the notion that community is a crucial

element for learning and thus, for effective education (Gunawardena & McIsaac, 2004;

Moore, 1993; Strijbos, Martens, & Jochems, 2004). Applied to the science classroom,

sense of community is foundational and related to gaining science literacy.

Little research has explored the effect of face-to-face collaborative learning versus

online collaborative learning on students’ sense of community (Koh & Hill, 2009), in

particular at the adolescent level and in the area of science. The majority of studies have

focused on university students’ sense of community in distance learning programs

(Cameron, Morgan, Williams, & Kostelecky, 2009; Koh & Hill, 2009; Ouzts, 2006) and

teachers’ sense of community in face-to-face interactions (Admiraal & Lockhorst; Baker

& Murray, 2011), thus leaving an integral gap in understanding the effects of

collaboration on adolescent students’ sense of community in the traditional and online

science classroom (Chiessi et al., 2010). Thus, this study sought to fill the gap in

examining the effect of online collaborative learning on adolescent students’ science

literacy and sense of community.

Problem Statement

A multitude of research exists that demonstrates the positive effect of

collaborative learning on student achievement as well as the effect of computer-mediated

or online collaborative learning on student achievement (Bluic et al., 2010; Winters &

Alexander, 2011; Yang & Chang, 2012). However, benefits of engaging in face-to-face

collaborative learning or online collaborative learning appear to differ depending on the

activity and the degree of interaction among students, leading to questions of how to best

20

foster student interaction in both traditional and computer-mediated learning

environments (Tutty & Klein, 2008).

The recent pedagogical push towards collaborative learning has shed light on the

importance of sense of community in the classroom as well (Wighting, Nisbet, &

Spaulding, 2009). However, the majority of studies have focused on the effects of

collaboration on university students’ sense of community (Cameron et al., 2009; Koh &

Hill, 2009; Ouzts, 2006). Little research exists that examines the effects of collaborative

learning on adolescent students’ sense of community.

The lack of science literacy and the related role of misconceptions of science

among adolescents has also been repeatedly documented (Harvard, 2011; Organisation

for Economic Co-operation and Development, 2007). Since science is generally

perceived as a collaborative subject (Scott et al., 2007), there is a growing need to address

the effect of online collaborative learning on science literacy as well as the effect of

online collaboration on sense of community. Utilizing the conceptual frameworks of

constructivism, social development theory, and communities of practice, this study

sought to determine the effects of online collaborative learning on middle school

students’ science literacy as measured by the Misconceptions-Oriented Standards-Based

Resources for Teachers (MOSART) assessment (Harvard, 2011) and sense of community

as measured by Rovai’s (2002a) Classroom Community Scale.

Purpose Statement

The purpose of this quasi-experimental, pretest/posttest control group design

study was to examine the effect of online collaborative learning on middle school

students’ science literacy and sense of community in a rural public school district in

21

South-Central Virginia. The theory of constructivism, social development theory, and

communities of practice theory formed the framework of this study.

The independent variable was the type of learning (traditional face-to-face

collaboration or online collaborative learning). Traditional learning was defined as

learning that occurs face-to-face in the classroom. Collaborative learning was generally

defined as learning that occurs as part of a group where all learners are mutually involved

in the learning process (Bernard, Rubalcava, & St-Pierre, 2000). More specifically,

online collaborative learning was operationally defined as computer-mediated learning

that occurs as part of a group where all learners are mutually involved in the learning

process (Dewiyanti et al., 2007).

The dependent variables were student science literacy and student sense of

community. Science literacy was generally defined as “understanding key scientific

concepts and frameworks, the methods by which science builds explanations based on

evidence, and how to critically assess scientific claims and make decisions based on this

knowledge” (Impey et al., 2011, p. 34), with a specific focus on the identification of

scientific misconceptions (Harvard, 2011). Sense of community was generally defined as

“a feeling that members have of belonging, a feeling that members matter to one another

and to the group, and a shared faith that members’ needs will be met through their

commitment to be together” (McMillan & Chavis, 1986, p. 9). The control variable,

student prior knowledge, was statistically controlled through the use of an ANCOVA.

Prior sense of community was measured but was not controlled for statistically as the two

groups did not demonstrate significant differences in their sense of community prior to

22

treatment. The intervening variables, gender and ethnicity, were controlled for by the use

of homogenous groups.

Significance of the Study

A need for research in science literacy that translates into practice in the

classroom through practical strategies has been documented in the educational research

(Anderson, 2007a); thus this study was both significant and timely. More specifically, no

current research exists that explores the effect of face-to-face collaborative or online

collaborative learning on science literacy, although it is widely accepted that science

knowledge is attained through social constructivist activities that may be provided by

peer-to-peer collaboration (Scott, Asoko, & Leach, 2007). With the recent widespread

reporting and resulting concern over lack of science literacy among adolescents, as well

as the growing ecological need for science literacy, this study was timely (Impey et al.,

2011; Organisation for Economic Co-operation and Development, 2007). Furthermore,

little research exists that examines the effect of online collaborative learning on sense of

community despite policymakers and educators alike who continue to push social

construction of knowledge inside and outside the classroom (Virginia Department of

Education, 2011b). In addition, the formation of social communities that requires

feelings of mutual care, respect, and work for the common good is becoming increasingly

frequent in pedagogy (Cheung, Chui, & Lee, 2011; Dewiyanti et al., 2007; Donne, 2012;

Yang & Chang, 2012). The results of this study have aided in filling the current gap in

the literature by testing the theories of constructivism, social development theory, and

communities of practice theory.

23

The results of this study were specifically relevant to central Virginia as

policymakers consider the adoption and revision of national standards such as the

Common Core State Standards for English Language Arts and Literacy in History/Social

Studies, Science, and Technical Subjects (Common, 2012; Virginia Department of

Education, 2011b). In addition, recent revisions to the state-mandated Virginia Standards

of Learning Science Standards have shown a migration towards learning that emphasizes

student inquiry, reflection, and collaborative learning (Luft, Bell, & Gess-Newsome,

2008; Virginia Department of Education, 2011a). Added to the current push to increase

technology use in the classroom (Virginia Department of Education, 2011a), research that

explores science literacy, collaboration, technology integration, and sense of community

are timely and much needed to ensure that new policy is research-based.

Perhaps of greatest importance was the significance of this study to teachers and

current pedagogy. As online group work becomes more popular (Koh & Hill, 2009) and

a need for practical strategies of increasing technology implementation (Witney &

Smallbone, 2011) and science learning grows (Anderson, 2007a), it is important that

teachers understand the inherent advantages and disadvantages to student collaborative

opportunities and the effects of science literacy and sense of community in both face-to-

face and online collaborative learning activities. This study provides increased

knowledge to teachers that may lead to more effective strategies to increase student

science literacy and sense of community at the adolescent level.

24

Research Questions

The following research questions guided this study:

Research Question 1: Is there a statistically significant difference in middle

school students’ misconceptions, an aspect of science literacy, as measured by the

MOSART testing instrument when participating in online collaborative learning as

compared to students who participate in traditional collaborative learning only?


school students’ sense of community as measured by the Classroom Community Scale

when participating in online collaborative learning as compared to students who

participate in traditional collaborative learning only?

Hypotheses

The following were the research hypotheses:

H1: There is a statistically significant difference in middle school students’

misconceptions of science as measured by the MOSART testing instrument when

participating in online collaborative learning as compared to students who participate in

traditional collaborative learning only, while controlling for student prior knowledge.


overall sense of community as measured by the Classroom Community Scale when


traditional collaborative learning only.


connectedness as measured by the Classroom Community Scale when participating in

25

online collaborative learning as compared to students who participate in traditional

collaborative learning only.


learning as measured by the Classroom Community Scale when participating in online

collaborative learning as compared to students who participate in traditional collaborative

learning only, while controlling for student community.

Alternatively, the following were the null hypotheses:

Ho1: There is no statistically significant difference in middle school students’



traditional collaborative learning only while controlling for student prior knowledge.





H03: There is no statistically significant difference in middle school students’







learning only.

26

Identification of Variables

The independent variable in this study was the type of learning. The two levels of

learning included traditional collaborative learning and online collaborative learning.

Traditional learning was defined as learning that occurs face-to-face in the classroom.

Online collaboration took place through the use of the Edmodo educational platform and

was defined as computer-mediated learning that occurs as part of a group in which all

learners are mutually involved in the learning process (Dewiyanti et al., 2007).

This study included two dependent variables. The first dependent variable,

science literacy, was defined as “understanding key scientific concepts and frameworks,

the methods by which science builds explanations based on evidence, and how to

critically assess scientific claims and make decisions based on this knowledge” (Impey et

al., 2011, p. 34). The MOSART (Harvard, 2011) assessment was used to measure

student science literacy. Developed by the Science Education Department of the

Harvard-Smithsonian Center for Astrophysics (Smith, n.d.), MOSART uses a variety of

multiple choice questions divided by grade level and specific subject area that have been

developed over five years by leading science and literacy experts (Harvard-Smithsonian

Center for Astrophysics, 2012; Center for School Reform at TERC, 2012). The questions

have been pilot tested and field tested for reliability and validity and were aligned with

the K-12 National Science Standards (Harvard-Smithsonian Center for Astrophysics,

2012). The MOSART Physical Science assessment, specifically, was used for this study.

The second dependent variable, sense of community, was defined as “a feeling

that members have of belonging, a feeling that members matter to one another and to the

group, and a shared faith that members’ needs will be met through their commitment to

27

be together” (McMillan & Chavis, 1986, p. 9). The Classroom Community Scale (Rovai,

2002a) was used to measure students’ sense of community. The Classroom Community

Scale consists of 20 items that ask students to describe their feelings on a five point

Likert-type scale (strongly agree, agree, neutral, disagree, and strongly agree) with two

subscales: connectedness and learning (Rovai, 2002a). Connectedness was defined as

“the feelings of the community of students regarding their connectedness, cohesion,

spirit, trust, and interdependence” (Rovai, 2002a, p. 206). Learning was defined as the

“feelings of community members regarding interaction with each other as they pursue the

construction of understanding and the degree to which members share values and beliefs

concerning the extent to which their educational goals and expectations are being

satisfied” (Rovai, 2002a, p. 206). The Classroom Community Scale has been field tested

for reliability and validity.

Finally, the control variable, student prior knowledge, is information acquired by

the student through the students’ history (Campbell & Stanley, 1963) and past

experiences (Gall, Gall, & Borg, 2007). Specific to this study, student prior knowledge

involved scientific misconceptions as measured by the MOSART assessment. Student

prior knowledge was statistically controlled through the use of an ANCOVA with a

pretest and posttest design (Gall et al., 2007). Prior sense of community was measured

using the Classroom Community Scale but was not controlled for statistically as the two

groups did not demonstrate significant differences in their sense of community prior to

treatment. The use of a pretest-posttest control group design ensured that the

“experiences of the experimental and control groups [were] as identical as possible” (Gall

et al., 2007, p. 405).

28

Definitions

Classroom Community Scale is a survey instrument developed by Rovai (2002a)

that measures the sense of community an individual perceives in relation to the classroom

and his or her classmates.

Collaborative learning is “the mutual engagement of learners in the learning

process rather than on the sole division of labour to reach a common group goal”

(Bernard et al., 2000, p. 262).

Connectedness is “the feelings of the community of students regarding their

connectedness, cohesion, spirit, trust, and interdependence” (Rovai, 2002a, p. 206),

measured as a subscale of the Classroom Community Scale.

Edmodo is an educational learning platform that allows social networking for

teacher and student connection and collaboration (Edmodo, 2012).

Ethnicity is an individual’s “origin of birth or descent…relating to race or culture”

(Jewell, 2002, p. 270).

Gender is “a person’s sex” (Jewell, 2002, p. 334), male or female.

Learning as a subscale of the Classroom Community Scale is the “feelings of

community members regarding interaction with each other as they pursue the

construction of understanding and the degree to which members share values and beliefs

concerning the extent to which their educational goals and expectations are being

satisfied” (Rovai, 2002a, p. 206).

Misconceptions are science conceptions that are held by students that are different

from scientific knowledge that is currently accepted within the science community at

large (Burgoon et al., 2011).

29

MOSART is Misconceptions-Oriented Standards-Based Resources for Teachers, a

group of assessments that enable educators to assess student level of science knowledge

through identification of scientific misconceptions (Harvard, 2011).

Online collaborative learning is a method of learning in which “the computer

facilitates interactions among learners for [shared] acquisition of knowledge, skills, and

attitudes” (Dewiyanti et al., 2007, p. 497).

Science literacy is the “understanding [of] key scientific concepts and

frameworks, the methods by which science builds explanations based on evidence, and

how to critically assess scientific claims and make decisions based on this knowledge”

(Impey et al., 2011, p. 34) as measured by the MOSART assessments (Harvard, 2011).

Sense of community is “a feeling that members have of belonging, a feeling that

members matter to one another and to the group, and a shared faith that members’ needs

will be met through their commitment to be together” (McMillan & Chavis, 1986, p. 9) as

measured by the Classroom Community Scale (Rovai, 2002a).

Student prior knowledge is information acquired by the student through the

students’ history (Campbell & Stanley, 1963) and past experiences (Gall et al., 2007).

Traditional learning is defined as learning that occurs face-to-face in the

classroom.

In the next section, a review of educational literature is provided. This will

include an explanation of educational learning theories, historical trends in education,

current trends in education, and recent educational reform initiatives. Additionally, the

aspects of science literacy and sense of community are discussed, including an in-depth

overview of assessments for each construct respectively. Finally, the current gap within

30

the literature that this study sought to fill are examined.

31

CHAPTER TWO: REVIEW OF THE LITERATURE

Introduction

Recent educational reform initiatives such as the Common Core State Standards

Initiative (Common, 2012), the National Science Education Standards (National Science

Teachers Association, 2012), and Science, Technology, Engineering, and Math (STEM)

(Stage & Kinzie, 2009) education have brought science learning to the forefront of

education (Virginia Department of Education, 2011b). In higher education institutions,

teachers are being introduced to new curricula and teaching strategies to increase

students’ science learning (Stage & Kinzie, 2009). The National Science Teachers

Association’s efforts to implement the National Science Education Standards in K-12

classrooms nationwide has resulted in creating new curricula and identifying more

effective teaching strategies to promote science literacy (Harvard, 2011; National

Science Teachers Association, 2012; Organisation for Economic Co-operation and

Development, 2007; Roberts, 2007).

Moreover, the unique characteristics of the current generation of learners (Black,

2010; Evans & Forbes, 2012) have led to a shift in students’ learning preferences

(Robinson & Stubberud, 2012), which has in turn influenced educational pedagogy

(Chelliah & Clarke, 2011). Students today are more socially connected through

technology than in generations past (Robinson & Stubberud, 2012). Evidence suggests

that adolescents prefer to learn through the use of technology that enhances

communication and collaboration (Elsaadani, 2012) and decreases response time

(Lightfoot, 2009). Research has supported that “students working in small groups tend to

learn more of what is taught and retain it longer than when the same content is presented

32

in other instructional formats” (Vaughan, Nickle, Silovs, & Zimmer, 2011, p. 113),

leading to a preference towards collaborative activities across subject areas (Bell et al.,

2010). More specifically, collaboration in the science classroom coupled with the use of

computer-supported collaborative learning has become increasingly popular (Bell at al.,

2010). Despite its popularity, researchers and practitioners are just beginning to examine

what constitutes quality and effective computer-supported collaborative science learning

(Clary & Wandersee, 2012).

There is a long history establishing the importance of considering technologies for

effective learning (Cobb, 1997; Kozma, 1994; Ullmer, 1994). Some suggest that

collaborative web-based technology can enhance learning, whereas others suggest that

the use of web-based technology cannot mimic what is done face–to-face and actually

detracts from collaboration among students. After examining effective web-based

learning, in higher education where most of the research has been conducted, Thomas

(2002) suggested that “the attainment of a discourse that is both interactive and academic

in nature is difficult within the online environment” (p. 359). On the other hand, Bonk

(2009) noted that web-based, collaborative technologies have the ability to transform the

manner in which education is delivered and can enhance learning.

Today’s learners have been labeled as “the most socialized generation in the

digital world” (Black, 2010, p. 96). The characteristic of increased socialization has led

to an increased need to consider how collaborative activities and technology

implementation may affect learning in the science classroom. A call for empirical

research that utilizes a theoretical framework to examine, technologies and modes of

delivery to improve the quality and effectiveness of web-based education exists

33

(Arbaugh, et al., 2008; Garrison & Kanuka, 2004; Song et al., 2004; Thompson &

MacDonald, 2005). A growing number of states are developing online K-12 programs to

provide increased options for the already over-stressed educational system (Ronsisvalle

& Watkins, 2005). Thus, as most of the literature on the delivery of web-based and

online education has been focused on higher education, a growing need exists to examine

the transferability of research findings on the K-12 population (Ronsisvalle & Watkins,

2005).

In distance education literature, two desired outcomes of educational experiences

have been identified: (a) meaning construction, also defined as critical thinking and deep

learning, and (b) construction of a collaborative community of learners (Garrison &

Anderson, 2003). Sense of community in the classroom is how students perceive that

they belong and matter to peers within a group (McMillan & Chavis, 1986). Since online

collaboration has been found to increase sense of community with undergraduate

students, specifically togetherness and the building of team bonds (Koh & Lim, 2012),

the effect of collaboration on sense of community at the secondary level warrants further

study. In addition, critical thinking has been recognized as a key component to effective

web-based learning (Garrison & Anderson, 2003) and can be likened to attaining science

literacy which requires that students apply critical thinking skills to scientific habits of

mind (AAAS, 1990).

This chapter, therefore, provides an overview of the current literature regarding

science literacy, sense of community, and collaborative learning. Collaborative learning

is discussed, including the definition, the underlying learning theory of social

development, and the relationship of collaborative learning to students’ learning.

34

Computer-mediated learning and web-based learning are discussed, with a specific focus

on the advantages of technology implementation inside and outside of the classroom.

Then, sense of community is reviewed, including a definition of sense of community and

the underlying learning theory, communities of practice, as well as the relationship

between sense of community and student learning. Science literacy is discussed,

including a specific operational definition of science literacy, the importance of science

literacy, and the levels of science literacy among adolescents nationally and worldwide.

Finally, an overview of the current literature gap and need for research is provided,

leading to the significance and implications of the current study.

Social Development Theory

The shared process of learning that emphasizes the importance of social

interactions is the foundation of social learning theory (Wenger, 1998; Wenger et al.,

2009). Specifically, social development theory posits that individuals learn by the

influence of others and through social relationships with others (Vygotsky, 1978). The

central tenet focuses on the idea that students learn not only through authentic activities

(a constructivist approach) but through social activities (Vygotsky, 1978; Yang & Chang,

2012) that require the engagement of dialogue to assist in problem solving. Vygotsky

(1978) proposed that individuals influence the environment surrounding them and are

also influenced by their environment. Likewise, individuals develop through the process

of collaborative learning (Vygotsky, 1978). The constructivist approach of social

development theory is central to the core of collaborative learning. Thus, any strategy

that enables increased collaboration, including computer-mediated technologies, may

35

encourage the development of social relationships (Baker-Doyle & Yoon, 2011;

Minocha, 2009b).

Social learning theory has roots in constructivism, which posits that mental

structures are constructed by the individual in response to interactions with the

environment (Dewey, 1997; Wenger, 1998). Constructivism suggests that individuals

learn by doing (Dewey, 1922, 1997); therefore, authentic activities that encourage learner

engagement provide greater learning than passive activities because they enable the

learner to construct his or her own knowledge (Vygotsky, 1978). As proposed by Dewey

(1922), education is the means of “social continuity of life” (p. 2) in which shared

activities produce knowledge (Peterson, Divitini, & Chabert, 2009). As Vygotsky’s

(1978) Zone of Proximal Development supports, language is critical to cognitive

development. Communication is imperative to effective learning (Vygotsky, 1978), and

learning should consist of constant communication, peer-to-peer and peer-instructor

(Peterson et al., 2009). Learning, therefore, is fundamentally a social phenomenon

enhanced through communication and group activity (Wenger, 1998; Wenger et al.,

2009).

Communication and shared activities can promote collaboration and thus

community and learning both in a traditional face-to-face classroom as well as in a

computer mediated environment. Collaboration and interaction among students and

teachers within the classroom creates a face-to-face community of learners (Peterson et

al., 2009). Communication is enhanced through many of today’s modern technologies

(Siemens, 2006). Social technologies such as blogs have been shown to provide a

flexible environment conducive to multi-way communication that allows discussion and

36

reflection while promoting interactivity that can increase learning relationships (Peterson

et al., 2009). Hence, a social constructivist approach to learning is supported by social

software tools that engage students in communication and collaboration which foster

critical thinking and problem solving (Minocha, 2009b), as well as gather information

from the ideas and experiences of others through networks of learning (Siemens, 2006).

When these collaborative relationships are fostered in the computer-mediated

setting, social networks are created. Social networks consist of the relationships that

individuals make with others that provide support and opportunities for growth and

enlightenment (Baker-Doyle & Yoon, 2011). Individuals inherently desire group

formation, especially when being a part of a group enhances experience and learning—an

aspiration that is enhanced by social networking (Minocha, 2009b). The desire to learn

while specifically making use of technology to contribute to society is supported by the

theory of connectivism. Connectivism posits that individuals desire to engage in learning

activities that assist in making sense of the world, developing personally, and

contributing to society as a whole through the use of technology, thus creating a network

of knowledge (Siemens, 2006). Creation of networks of knowledge requires

externalization of ideas expressed through communication. As a result, social

networking potentially increases opportunities for collaborative learning through

discussion that transcends time and geographic barriers (Lim, Yang, & Zhong, 2009;

Siemens, 2006).

An increased amount of learning is now taking place outside of the classroom

walls, facilitated through the use of technology (Siemens, 2006), thus creating learning

environments that are conducive to more collaborative and interactive activities that

37

enable the construction of knowledge cooperatively (Peterson et al., 2009). Most

importantly, social development theory supports that learning is a social activity

(Nuankhieo et al., 2007; Peterson et al., 2009). Cognitive development, as supported by

social development theory, occurs through dynamic and complex interactions with

mature members of society (Sivan, 1986). In this case, being more mature has less to do

with age but rather with being knowledgeable (Vygotsky, 1978). As a result, students

can benefit from a relationship of reciprocity between peers (Ding & Harskamp, 2011)

and the engagement in social networks of learning (Siemens, 2006).

In the face-to-face environment, collaborative activities have been shown to foster

inquiry, critical thinking, and intellectual development (Ding & Harskamp, 2011). In the

science classroom, collaborative laboratory learning has been shown to increase interest,

curiosity, and motivation through hands-on and social learning (Ding & Harskamp,

2011). Creation of social learning opportunities in the classroom have also been shown

to support effective communication, sharing of knowledge, and learner satisfaction (Tutty

& Klein, 2007).

Social development theory, therefore, encompasses the very nature of learning

through peer assistance, where individual needs and goals are met through instructional

and motivational constructs (Sivan, 1986). Since collaboration has been shown to

increase cognitive knowledge, affective knowledge, and motivation (Saleh, Lazonder, &

Jong, 2007; Stump, Hilpert, Husman, Chung, & Kim, 2011; Yang & Chang, 2012), it can

be considered an assistive technique inherent to social development theory that may also

assist in building learning communities.

38

Collaborative Learning

Collaborative learning, or “the mutual engagement of learners in the learning

process rather than on the sole division of labour [sic] to reach a common group goal”

(Bernard et al., 2000, p. 262), has become an increasingly popular teaching strategy

across the globe (Johnson & Johnson, 1996; Moolenaar, Sleegers, & Daly, 2012) both in

the preK-12 setting (Johnson & Johnson, 2009) and the post-secondary setting (Bell et

al., 2010; Vaughan et al., 2011). Unlike many educational practices that tend to come

and go as fads, collaborative learning has remained a preferred pedagogical practice since

the 1980s (Johnson & Johnson, 2009). This is due in part to the benefits that

collaborative learning has to offer for teaching and learning as supported in numerous

empirical studies (Ding & Harskamp, 2011; Miller & Benz, 2008; Parveen & Batool,

2012; Zhu, 2012).

Research studies have found collaboration to provide benefits to teaching and

learning, including increasing students’ motivation, feelings of success, mutual

interdependence (Miller & Benz, 2008), communication, level of satisfaction (Zhu,

2012), cognitive growth, and socio-emotional or affective growth (Parveen & Batool,

2012). Collaboration has been shown to be an effective educational practice in meeting

the needs of a diverse array of learners with differing needs, personalities, experiences,

goals, and levels (Miller & Benz, 2008) by providing opportunities for differentiation in

both instruction and learning.

Collaboration fosters engagement in an active learning process, increasing the

likelihood of knowledge acquisition and transfer (Treagust, 2007). Research has shown

that constructive learning processes are enhanced through collaborative learning

39

(Dewiyanti et al., 2007). Collaboration also provides opportunities for inquiry-based

learning, which has been shown to increase long-term knowledge retention (Akinbobola

& Afolabi, 2009).

Inquiry-based learning requires a shift from teacher-centered learning to student-

centered learning (Luft et al., 2008). Collaborative learning provides this shift, allowing

students to become the focus of instruction through group responsibility (Dewiyanti et al.,

2007). As collaborative partnerships are often necessary in the successful workplace, the

acquisition of skills that promote inquiry and critical thinking are imperative. These

skills are fostered through active learning that promotes engagement with others (Stump

et al., 2011). Since collaborative learning is an active learning process (Chelliah &

Clarke 2011), collaborative teaching techniques may lead to increased peer engagement

(Moore, 2011). Active learning has been shown to produce increased persistence, more

positive student attitudes, and greater student achievement than passive learning (Stump

et al., 2011), thus becoming a more desirable learning strategy (Cheung et al., 2011).

Despite the advantages provided by active learning, educational pedagogy has been slow

to change; therefore, a further understanding of learning strategies, such as collaborative

learning, is needed to influence future pedagogy.

Collaborative learning that occurs in small peer groups may assist students in

communication, sharing ideas, and obtaining feedback from peers, thus leading to gains

in student achievement and meaningful knowledge building (Stump et al., 2011).

Collaborative group techniques allow students to engage in discussion and debate, where

the learner must not only defend his viewpoint but also consider the ideas and opinions of

others (Ding & Harskamp, 2011). Individuals more effectively construct their own

40

meaning when allowed opportunities to collaborate with others through the process of

critique, defense, and justification of concepts and opinions (Stump et al., 2011).

Discussion within groups can assist lower-achieving students in increasing active

learning through group participation (Saleh et al., 2007).

In collaborative learning, students work together for the learning of both the

individual and community (Parveen, Mahmood, Mahmood, & Arif, 2011). Collaborative

learning has been shown to be beneficial in partnerships where an asymmetry of

knowledge exists, allowing peers to both provide and gain knowledge in a reciprocal-type

relationship (Saleh et al., 2007; Stump et al., 2011). As participation increases,

collaborative learning in small groups can lead to increased student learning (Saleh et al.,

2007).

Studies have shown that collaborative learning strategies in higher education

classrooms increase student motivation (Saleh et al., 2007), positive attitudes towards

learning (Yang & Chang, 2012), and academic achievement (Yang & Chang, 2012).

Research has also shown that student engagement in collaborative learning with peers

increases student interest and confidence in the science classroom (Ding & Harskamp,

2011). With increased student interest, engagement, and motivation, construction of

conceptual understanding of scientific concepts may improve (Cavas, 2011).

Collaboration in the Science Classroom

Collaboration has been found to be especially effective in the science classroom,

as students have the opportunity to experience science as an active process and a common

endeavor that yields information about the true nature of science (Treagust, 2007). This

is in part due to the nature of laboratory learning that lends itself to collaborative group

41

learning (Ding & Harskamp, 2011). Recently, an increased push for inquiry-based

learning in the science classroom (Luft et al., 2008) has become a national imperative

(National Science Teachers Association, 2012). Laboratory learning, or learning that

occurs through hands-on experimentation and modeling, has been shown to encourage

inquiry and increase intellectual development, thus assisting in formation of scientific

concepts that relate to the real world (Ding & Harskamp, 2011). Active and collaborative

engagement is fostered when students are allowed opportunities to participate in inquiry-

based activities that employ scientific methods (Fang & Wei, 2010) much like scientists

in the real world. Like scientists, students are able to share ideas and build upon one

another’s work (Luft et al., 2008, Lunetta et al., 2007). Science, therefore, is a social and

collaborative practice that promotes sharing and learning among peers (Kelly, 2007;

Miller & Benz, 2008). Collaborative learning techniques such as participation in group

laboratory activities enhance science learning (Parveen & Batool, 2012; Parveen et al.,

2011) by fostering higher order thinking skills allow students to engage in critical

problem-solving and enhancing connections to real-life situations.

Despite the popularity of laboratory and authentic hands-on learning in the

science classroom, many instructors find it difficult, with limited class time, to

incorporate laboratory activities that foster higher order thinking, such as integrating

problem-solving skills and making connections with real-life situations (Ding &

Harskamp, 2011). Furthermore, traditional laboratory activities in the science classroom

tend to provide large amounts of information that some students may find difficult to

digest in a short period of time (Ding & Harskamp, 2011). However, the information

provided in class, without authentic learning opportunities, often consists of rote

42

memorization of facts which have been found to be insufficient in supporting science

learning (Akinbobola & Afolabi, 2009; Fata-Hartley, 2011).

Given the socially constructed nature of science learning (Anderson, 2007b), the

advancement in technology may be beneficial to student science achievement (Songer,

2007). The integration of technology may enhance teaching and learning and be able to

address some of the concerns related to authentic and laboratory learning in the science

classroom. However, a need exists to study what type of collaborative activities and what

type of technology support science learning (Songer, 2007), especially given that

different features of “CMC technologies may support different types of tasks and learners

in different ways” (Zhao, Alvarez-Torres, Smith, & Tan, 2004, p. 46). Additionally,

some educators report limitations of technology use in the classroom, which may result in

decreased quality of learning (Zhao et al., 2004). Given the focus of recent research of

online learning in higher education, a growing need exists to examine the possible

advantages and disadvantages of computer-mediated learning in the K-12 setting

(Ronsisvalle & Watkins, 2005).

Computer-Mediated and Web-based Collaborative Learning

The increasing use of technology in the workplace, home, and school has led to a

shift in how individuals learn and how information is delivered (Chelliah & Clarke, 2011;

Koh & Lim, 2012). Collaboration in a technological world, therefore, has become a

necessity (Wang, 2010). The trend in the information technology world is producing a

switch from typically offline software to online software, thus increasing opportunities

for computer-mediated collaborative learning both in and out of the workplace (Koh &

Lim, 2012). The use of computer-mediated technology allows users the ability to

43

conduct work collaboratively (Koh & Lim, 2012; Wang, 2010). This is especially

important as professionals must have the skills and capability to effectively collaborate

within the field; in turn, educational institutions must prepare students for such situations

(Lim et al., 2009), which presents a need to integrate computer-mediated collaborative

activities in the classroom.

Learners of the 21st century, termed “Homo sapiens digital, or digital human”

(Prensky, 2009), were born into a technology-rich world (Black, 2010). The current

generation of learners “live, work, and study in technology rich cultures” (Chelliah &

Clarke, 2011, p. 277) and prefer new ways of accessing information quickly and

efficiently (Black, 2010). Today’s learners demand information in new and often

challenging ways—requiring information access, insisting on immediate feedback,

engaging in multi-tasking, and being connected to others almost constantly (Black, 2010;

Evans & Forbes, 2012; Wenger et al., 2009). All of the aforementioned learner

characteristics are driven by web-based technologies (Black, 2010).

Web-based technologies are also known as social software tools—computer-

mediated tools that enable interaction and sharing with others (Minocha, 2009a). With

computer-mediated tools, individuals cooperatively create and share knowledge, fostering

active participation in the learning process (Minocha, 2009a). While numerous social

computer-mediated tools exist, many are used not only in the social realm, but in the

educational realm as well. These technologies include Facebook, Twitter, and YouTube

(Minocha, 2009a) and may provide benefits for students and teachers alike (Minocha,

2009a; Wang, 2010). Edmodo, one such educational platform, provides opportunities for

students and teachers to engage in social networking activities, thus providing safe

44

environments for sharing ideas, asking questions, and collaborating on education-related

activities (Trust, 2012; Werner-Burke, Spohn, Spencer, Button, & Morral, 2012). The

social nature of the online learning environment created by Edmodo provides the

essential components of opportunities for student motivation and engagement required

for learning (Werner-Burke et al., 2012). Additionally, an environment that fosters

discussion and collaboration is provided, which some consider the cornerstone of the

online learning environment (Palloff & Pratt, 2005a).

One research study found that Edmodo was preferred by teachers over other

educational social networking platforms (Trust, 2012). Advantages reported included a

safe place for sharing ideas, asking questions, and collaborating with other educators.

Disadvantages reported were information overload, including the overwhelming amount

of information, the need to learn new social norms, and the need to learn new tools

(Trust, 2012). Another study involving middle school students found that Edmodo was

an effective tool for computer-facilitated written discussion (Werner-Burke et al., 2012).

Research suggests that the use of modern technologies may enhance the learning

process through construction of knowledge (Findlay, 2012; Zhu, 2012) and shared

meaning (Siemens, 2006). Computer-mediated collaborative learning can assist in the

development of problem-solving skills by establishing a collaborative learning

community (Clary & Wandersee, 2012), which may translate into future success in the

world of work (Minocha, 2009b), thus assisting in preparing students for successful

citizenship and global competency (Poore, 2011). Furthermore, use of social

technologies may foster interaction and assist in the creation of networks that facilitate

the sharing of knowledge and experiences (Siemens, 2006).

45

Computer-mediated collaborative learning has been reported to show many of the

same benefits as face-to-face collaboration, such as increased learning outcomes and

academic performance (Koh & Lim, 2012). The interactive nature of web-based

technologies bode well for collaborative and cooperative activities while fostering the

development of learning communities (Minocha, 2009b). The development of a learning

community may lead to mutual engagement, joint enterprise, and a shared repertoire that

fosters knowledge acquisition (Wenger, 1998; Wenger et al., 2009). Individuals are

given increased flexibility and opportunities to engage in learning relationships with

peers through the use of social software such as that provided by web-based technologies

(Minocha, 2009b; Siemens, 2006), therefore transcending time and geographic barriers

(Dawson, 2008), providing increased flexibility, convenience, opportunities for feedback,

and long-term knowledge retention (Lim et al., 2009).

As opportunities for feedback and interaction increase, students’ motivation to

learn, ability to retain information, and academic performance may also increase (Mahle,

2011). Collaborative learning, therefore, is facilitated by the social nature of web-based

technologies (Cheung et al., 2011). However, the majority of studies on online learning,

specifically collaborative learning, tend to focus on students in higher education

(Dewiyanti et al., 2007; Miller & Benz, 2008; Nicholas & Ng, 2009; Zhu, 2012).

Research suggests that technology is not suited for all learning tasks (Zhao et al., 2004).

While the incidence of technology integration in the classroom has increased, few

practices are evidence-based and may be ineffective for learning (Zhao et al., 2004). A

key component of computer-mediated learning, however, has been the extent and quality

of communication as well as the immediacy of feedback (Zhao et al., 2004). Thus, a

46

growing need exists to examine the impact of technologies on student learning at the

adolescent level (Resta & Laferriére, 2007; Songer, 2007) as well as the generalizability

of research findings to the K-12 classroom (Ronsisvalle & Watkins, 2004; Zhao et al.,

2004).

Community and Effective Online Learning

Imperative to examining the effects of collaboration on student learning is the

understanding of social presence, otherwise known as sense of community, or the

feelings of connection and community among peers (Palloff & Pratt, 2005b).

Collaboration has been shown to increase learner sense of community by reducing the

potential for learner isolation, therefore increasing opportunities for in-depth learning

experiences (Palloff & Pratt, 2005b). The creation of a learning community is cultivated

through collaboration that affords social construction of knowledge (Palloff & Pratt,

2000).

The Community of Inquiry (CoI) framework is perhaps one of the most

thoroughly researched and reported frameworks when studying online education

(Garrison & Arbaugh, 2007; Garrison, Anderson, & Archer, 2010). The CoI framework

is based on three elements: cognitive presence, social presence, and teaching presence

(Garrison, Anderson, & Archer, 2010). Cognitive presence is defined as “the extent to

which learners are able to construct and confirm meaning through sustained reflection

and discourse” (Garrison & Arbaugh, 2007, p. 161). Social presence is defined as the

“ability of learners to project themselves socially and emotionally, thereby being

perceived as ‘real people’ in mediated communication” (Garrison & Arbaugh, 2007, p.

159). Teaching presence is defined as the “design, facilitation, and direction of cognitive

47

and social processes for the purpose of realizing personally meaningful and educationally

worthwhile learning outcomes” (Garrison & Arbaugh, 2007, p. 163). The underlying

assumption of the theory is that effective online learning occurs when interactions

between the three essential elements overlap, thus becoming an indicator of the

effectiveness of online education.

Rovai (2002) suggested that the classroom community is a social community of

learners who learn through the sharing of knowledge, values, and goals. However, the

extent to which students experience feelings of disconnect may lead to decreased

participation in the learning community; thus, connectedness is an important indicator of

the effectiveness of the learning community. In order for learning to occur, feelings of

connectedness and community must occur to facilitate acceptance of group values and

goals (Rovai, 2002). Therefore, the extent to which the learner experiences learning and

connectedness will influence the learner’s sense of community—or social presence—and

thus influence the quality of the learning experience.

Communities of Practice Theory

Sense of community, the general feeling of belonging (McMillan & Chavis,

1986), is grounded in the theory of communities of practice (Wenger, 1998; Wenger et

al., 2009). The theory of communities of practice follows in the footsteps of social

development theory in that the central theme posits that learning is a fundamentally social

phenomenon that occurs through social participation (Wenger, 1998; Wenger et al.,

2009). Four premises exist within the theory of communities of practice: learners are

social beings, knowledge entails competence and respect for valued enterprises, knowing

occurs through active engagement, and learning should ultimately produce meaning

48

(Wenger, 1998). The four premises lay wake to four interconnected components

(meaning, practice, community, and identity) that support the idea that social

participation fosters the learning process (Wenger, 1998). These components are defined

as follows:

Meaning: a way of talking about our (changing) ability—individually

and collectively—to experience our life and the world as meaningful.

Practice: a way of talking about the shared historical and social

resources, frameworks, and perspectives that can sustain mutual

engagement in action.

Community: a way of talking about the social configurations in which

our enterprises are defined as worth pursuing and our participation is

recognizable as competence.

Identity: a way of talking about how learning changes who we are and

creates personal histories of becoming in the context of our

communities. (Wenger, 1998, p. 5)

A community of practice may be defined as community that develops over a

period of time through the pursuit of a common set of wants, needs, and goals. A

community of practice can manifest as any informal group in which mutual engagement,

joint enterprise, and a shared repertoire exist—a family, a work group, a class, or a social

group. Thus, a fundamental part of people’s daily lives involves communities of practice.

In order for true learning to occur, individuals must participate within a community of

practice where knowledge is socially gained (Wenger, 1998) through an interconnected

web of giving and receiving (Gardner, 1996). As learning occurs through the interactions

49

of daily life, learning is the result of both social structure and situated experience

(Wenger, 1998). Learning, therefore, consists of the shared histories and experiences of

the community of practice (Wenger, 1998).

Learning is not a static object, but rather an emergent, cycling process (Wenger,

1998). The practice of learning is an “ongoing, social, interactive process” (Wenger,

1998, p. 102). A community of practice engages learners through establishment of

personal identity, mutual engagement, and creation of complex interrelations between

new and existing members (Wenger, 1998). Participation within the community of

practice is essential for situated learning to occur (Smith, 2003). Situated learning posits

that learning occurs as a result of social learning as well as the surrounding culture that

influences mental functioning (Lave & Wenger, 1991).

Several principles support the idea of social learning within a community of

practice:

An intrinsic part of human nature is learning; therefore it is an inseparable and

ongoing life activity (Wenger, 1998).

Learning involves negotiation of new meanings; thus engagement and

participation is essential for learning to occur (Wenger, 1998).

Structure is created by learning; thus, implying that experience and continuous

negotiation of meaning are imperative for learning (Wenger, 1998).

Learning involves inherent experiential and social constructs; thus, supporting

the importance of social engagement, creation of social identity, and building

of experience within the community of practice (Wenger, 1998).

50

Each of those principles sustains the idea that learning is social and requires learner

participation. Thus, learning is fostered through educational processes that are situated in

social participation (Wenger, 1998). Understanding and enhancing social opportunities

for learning, such as that provided in collaborative activities, is essential in cultivating

citizens that are globally competent and able to function in today’s ever-changing society

(Wenger, 1998).

Research has shown that pedagogy rooted in social activities promotes learner

satisfaction (Rovai, Wighting, & Liu, 2005). As pedagogy increasingly begins to require

technology integration, examining the effects of multiple methods of instruction on the

learning community will become necessary (Rovai, 2002b). Perhaps more importantly,

research has found a relationship between peer connectedness within the learning

community and cognitive learning, thus suggesting that activities that foster social

learning within a community of practice may increase sense of community (Rovai,

2002b). An understanding of the social bonds that are created among learners in both

face-to-face and computer-mediated learning is necessary in order to develop a more

comprehensive picture of sense of community in the adolescent classroom (Rovai et al.,

2005).

Technology has been purported to provide links for common experiences among

individuals, thus establishing a community of practice (Wenger, White, & Smith, 2009).

As a community of practice is established, learning may become more relevant, trust and

mutual engagement may increase, and communicative learning may therefore be fostered

(Wenger et al., 2009). As technologies that enable communication become more

prevalent, the applicability of communities of practice theory increases as communities of

51

practice is centered on the potential of individuals to work together in learning

communities (Wenger et al., 2009). Technology therefore provides new opportunities for

community and the creation of digital habitats--the intersections of technology and

community (Wenger et al., 2009). As digital habitats are created and users experience

increased participation and engagement, individual and group identity may be

encouraged. Certain practices may be formed that bridge interaction with and between

technologies, which may create community agreements (Wenger et al., 2009), thus

leading to the construction of sense of community.

Sense of Community

Sense of community is defined as “a feeling that members have of belonging, a

feeling that members matter to one another and to the group, and a shared faith that

members’ needs will be met through their commitment to be together” (McMillan &

Chavis, 1986, p. 9). Sense of community is generally considered to have historically

rural roots (Glynn, 1981) based on a geographical location (Palloff & Pratt, 1999). From

the times when members of a town would gather to hunt, farm, gather, and feed, the

formation of a community, and hence a sense of community, was not a conscious

process. These early communities demonstrated several qualities: homogeneity,

interdependence, shared responsibility, face-to-face relationships, and common goals

(Glynn, 1981; Palloff & Pratt, 1999). The development of a sense of community was

essentially a matter of survival; therefore, a high level of sense of community was

considered beneficial to society (Glynn, 1981).

Over time, however, sense of community has purportedly declined (Glynn, 1981).

This is attributed in part to the breakdown of social supports that have been an inherent

52

part of communities (Abfalter, et al., 2012; Glynn, 1981). Along with the loss of sense of

community, society has experienced loss of autonomy, decreased involvement in the

community, and loss of relationships that foster mutual growth and sustainment, thus

leading to a decrease and breakdown of integral support systems (Glynn, 1981) and,

therefore, sense of community (Koh & Hill, 2009). The efforts to increase opportunities

for the building of sense of community, especially among young adults, often times has

fallen short of social needs and expectations (Abfalter et al., 2012), thus leading to a need

to understand more fully practices that encourage and foster sense of community

(Cameron et al., 2009).

Research has shown that sense of community is more than an abstract concept

(Glynn, 1981). Glynn (1981) found that sense of community may be defined as a group

of attitudes and behaviors that provide a specific set of characteristics for a given

community, namely characteristics related to satisfaction and competence, which are

essential in development of a mutually responsive community (Glynn, 1981). This

supported earlier findings by Hillery (1955) that concluded that the concept of

community was rooted in social interaction regardless of geographic situation, which has

since been supported by current study (Zhao et al., 2012). Recent research has found that

sense of community fosters social identity, a key component of learning (Chiessi et al.,

2010; Palloff & Pratt, 2005b), which increases the opportunities for learning within

schools (Admiraal & Lockhorst, 2012; Sancho & Cline, 2012).

Sense of community consists of four interacting elements: membership,

influence, integration and fulfillment of needs, and shared emotional connection (Abfalter

et al., 2012; McMillan & Chavis, 1986). Membership is defined as an individual’s

53

identification and sense of fitting in with other members of a specified group (Abfalter et

al., 2012; Palloff & Pratt, 1999). Influence is characterized by the not only the influence

of the community on the individual, but also the individual’s perceived impact on the

surrounding community (Abfalter et al., 2012). Integration and fulfillment of needs

entails the incentives, rewards, and reinforcements that are essential to becoming a

member of a community and maintaining the community (Abfalter et al., 2012; McMillan

& Chavis, 1986). Shared emotional connection involves the experiences, history, and

identification that members share with the community (Abfalter et al., 2012).

Given the multiple facets of sense of community, Rovai (2002b) identified two

overarching components of sense of community within the classroom: connectedness and

learning. Connectedness is categorized as interpersonal relationships and is fostered

through the building safety and trust among peers (Rovai, 2002b). Connectedness

denotes a certain level of care and satisfaction among group members which may lead o

development of a learning community (Rovai, 2002b). Likewise, learning is indicated as

the active and social construction of knowledge that results from a thriving learning

community (Rovai, 2002b). Learning is accomplished through the shared creation and

meeting of goals among members of the community (Rovai, 2002b).

While the definition of sense of community in early studies centered on the

interpersonal relationships and feelings of loyalty and belonging within a community

(e.g. town or neighborhood), society in the modern age has formed sense of community

focused more centrally on interests and skills (McMillan & Chavis, 1986). Thus, the

concept of community can be extended from a geographic location to a community

without spatial boundaries (Palloff & Pratt, 1999). This extends to the community within

54

specific disciplines or classrooms, forming modern-day face-to-face learning

communities (Buch & Spaulding, 2011; Chiessi et al., 2010).

In today’s technological world, a community without spatial boundaries entails

the concept of the virtual community, a growing topic in the business, social, and

educational world (Zhao, Lu, Wang, Chau, & Zhang, in press). Hagel and Armstrong

defined virtual community as community constructed through computer-mediated spaces

where communication encourages content that is generated not by individuals, but rather

by the overall community (as cited in Zhao et al., in press). Just as in a face-to-face

community, the component of belonging within the community is essential for positive

outcomes in the virtual community (Palloff & Pratt, 1999; Zhao et al., in press).

Research has found many benefits to individuals who possess increased sense of

community (Abfalter et al., 2012; Glynn, 1981; Yu et al., 2010). Sense of community

has been positively correlated with academic achievement (Wighting et al., 2009).

Wighting et al. (2009) also found a correlation between learning community and

academic achievement (Glynn, 1981). Research also states that involvement and

belonging in a community may provide benefits to adolescent development (Evans,

2007). Benefits to adolescents extend from development of personal identity, increased

participation in community activities, and formation of peer groups based on shared

characteristics (Chiessi et al., 2010; Pugh & Hart, 1999). The array of experiences

coupled with the quality of experiences are paramount to the experience of sense of

community, providing opportunities for adolescents to engage in influential and powerful

social roles—roles upon which relationships within society are developed (Chiessi et al.,

55

2010). Social experiences are essential to adolescent development (Evans, 2007) and

lead to the creation of a social learning community (Rovai, 2002b).

The interactions between individuals within a community, specifically a learning

community, are important in the development of both personal and social identity

(Chiessi et al., 2010). Therefore, a strong sense of community within a school may

positively impact both students and teachers (Rossi, 1997). Studies involving

undergraduate students have found a correlation between increased sense of community

and perceived academic achievement (Buraphadeja & Kumnuanta, 2011). At the high

school level, sense of community has been found to increase learning and academic

achievement by increasing peer involvement (Wighting et al., 2009). Despite current

knowledge of sense of community in the classroom that has focused on adult learners, a

need exists to examine sense of community more thoroughly among adolescents (Chiessi

et al., 2010; Evans, 2007: Wighting et al., 2009). Given that an essential component to

successful science communities is an elevated sense of belonging, a more in-depth

understanding of sense of community within the adolescent science classroom is

necessary (Hsu & Roth, 2010).

Review of Literature

Research has shown that students that engage in a community of practice within

the classroom and that experience increased sense of community develop interpersonal

relationships that foster mutual respect and furthering of collaborative goals (Kelly,

2011). Research has also supported that sense of community is essential for development

of personal identity and is a key component for the social and psychological well-being

of adolescents (Chiessi et al., 2010). When adolescents are participatory members of a

56

group, their emotional needs are often met through their perceived influence on the

community and their feelings of belonging (Chiessi et al., 2010). Individuals may come

to realization that the product of the community may far outweigh what could be

produced by the individual when a rich community exists, thus creating a sense of

community (Kelly, 2011). As community is increased within the classroom, the

increased interest in the mutual success of the group may transcend the school boundaries

and reach into the surrounding neighborhood (Roxas, 2011).

Research on communities of practice and sense of community among adolescents

in the traditional brick-and-mortar classroom has tended to focus on students of

diversity—refugees and students of differing cultures (Kelly, 2011; Roxas, 2011) and

students transitioning from one school to another (Sancho & Cline, 2012). Given the

social constructivist view that meaning, learning, and self-identity are forged through

social experiences (Vygotsky, 1978; Vygotsky, 1986), understanding how social

activities influence adolescent development is important and timely (Evans, 2007).

Research has supported that adolescents experience sense of community differently than

adults (Evans, 2007) and that sense of community is related to establishing a sense of

personal identity, especially in rural communities (Shamah, 2011). Research has also

established the important role that schools play in providing support, connection to peers,

and relationships with adult mentors (Shamah, 2011). Rural communities in particular

have exhibited the need for school-related activities that engage students with their peers,

where few opportunities may exist for social engagement otherwise (Shamah, 2011). An

increasing need subsists in understanding and identifying methods to foster sense of

community among adolescents in the classroom (Sancho & Cline, 2012; Shamah, 2011).

57

A recent study involving high school students found that increased collaboration

through problem-based learning in the general science classroom facilitated the building

of a learning community, thus increasing student sense of community (Ferreira & Trudel,

2012). Students participating in online collaborative group activities were found to have

an increased level of sense of community as compared to students participating in online

paired peer activities only (Ferreira & Trudel, 2012). Therefore, an increased level of

interaction may lead to increased learning, academic achievement, and sense of

community (Ferreira & Trudel, 2012).

Wighting et al. (2009) found a correlation between sense of community and

academic achievement among high school students when examining student achievement

as measured by the PSAT. A moderate correlation was also found between learning

community and academic achievement (Wighting et al., 2009). However, the study was

small-scale and may not be generalizable to other populations. Further suggestions for

study included examining the effect of sense of community on academic achievement

using a variety of measurement tools (Wighting et al., 2009).

A study involving middle and high school students and their teachers found that a

relationship existed between middle school student perceptions of relationships (peer-to-

peer and peer-to-teacher) and academic achievement (Schulte, Shanahan, Anderson, &

Sides, 2003). However, the same relationship was not found with high school students,

warranting further research. A relationship between sense of community, school

attendance, and academic achievement was also noted; however, further examination was

suggested (Schulte et al., 2003).

58

Buraphadeja and Kumnuanta (2011) found that increased collaboration through

peer tutoring cultivated shared values and beliefs, construction of knowledge, and

positive feelings towards peer-to-peer interaction. The enhancement of instruction

through such practices as technology implementation, including information

communication technologies (ICT) may also increase students’ sense of community

(Buraphadeja & Kumnuanta, 2011), although research indicates that some students may

find the formation of learning communities through technology more difficult and less

important than those formed in face-to-face courses (Cameron et al., 2009).

Currently, little research exists that examines community and the social context of

learning within the science classroom specifically (Anderson, 2007b; Fraser, 2007;

Lunetta et al., 2007). While science knowledge is socially (Anderson, 2007b) and

experientially (Atkin & Black, 2007) constructed, the bulk of the research is based on

theory rather than practice in the classroom (Roberts, 2007). Given the importance of

science literacy to making sound social decisions, an examination of sense of community

with goals motivated towards the common good is necessary (Roberts, 2007). Thus, a

gap exists within the literature to study sense of community among adolescents (Evans,

2007), the effects of implementation of technology in the classroom on sense of

community (Buraphadeja & Kumnaunat, 2011), the effect of community and learning

environment on science achievement (Fraser, 2007), and the effects of technology

implementation on science literacy (Bell, 2007).

Science Literacy

Science literacy has recently become an educational focus in the literature and in

educational reform around the world (Organisation, 2007; Roberts, 2007). Science

59

literacy is defined in many different ways (Roberts, 2007). Despite the multitude and

disparity among definitions, however, science literacy has come to mean education of all

students in the area of science; the overarching understanding of science in general rather

than a specific preparation for scientific careers (Roberts, 2007). This implies a specific

level of science content knowledge and demands increased science learning within

secondary education (Roberts, 2007).

Still, what constitutes science education differs widely from nation to nation. A

more thorough definition, therefore, of science literacy (and the operational definition

used in this study) is the “understanding [of] key scientific concepts and frameworks, the

methods by which science builds explanations based on evidence, and how to critically

assess scientific claims and make decisions based on this knowledge” (Impey et al., 2011,

p. 34). In short, science literacy has been deemed engagement of all individuals with

science (Roberts, 2007); the level of science knowledge and understanding that allows

application of scientific concepts to real-world issues.

Important to understanding the concept of science literacy is the idea of the

detriment caused by common scientific misconceptions (Harvard, 2011), as well as what

scientific knowledge learners must attain in order to become competent citizens (Roberts,

2007). Since today’s learners will become tomorrow’s scientists, science literacy also

emphasizes the need for lifelong learning (Liu, 2009; Roberts, 2007). The rapid advances

in science and technology seen in today’s world warrant a renewed interest in the science

literacy of all citizens in order to ensure continued economic development (Liu, 2009).

This renewed interest is evident in initiatives worldwide that call for student proficiency

in science literacy prior to graduation from high school (Liu, 2009).

60

The worldwide educational interest in science literacy can be traced to the 1950s

and 1960s, when science literacy was deemed necessary for students who would not enter

the scientific field post secondary graduation (Liu, 2009; Roberts, 2007). The United

States expressed a specific interest in science education through the National Defense

Education Act (NDEA), which was signed into law in 1958 and provided opportunities

for government grants to fund materials, technology equipment, and minor remodeling of

laboratory spaces for student science education in public and nonprofit private schools

(Institute for Defense Analyses, 2006). Science literacy was later reasoned in the 1970s

to be imperative for all students regardless of desired career path, ability, interest, and

background (Liu, 2009). Current opinions of science literacy demand proficiency for all

citizens (Liu, 2009; Roberts 2007), with a specific focus on youth who have been

documented to be lagging behind in scientific expectations (Organisation for Economic

Co-operation and Development, 2007). The importance of science literacy was

specifically recognized during the 1980s with the creation of the American Association

for the Advancement of Science’s Project 2061 in 1985 which identified areas necessary

for student understanding of science, mathematics, and technology and prescribed

benchmarks that American students should meet or exceed (AAAS, 1990).

Subsequently, the number of students participating in continued education increased, the

necessity of a scientific work force was noticed, and the need for a citizenry that

possessed certain scientific knowledge and skills was discerned (Roberts, 2007), thus

solidifying the need for increased science literacy among adolescents and adults in order

to participate in full and productive lives (AAAS, 1993).

61

With the recent advent of initiatives that push scientific learning specifically in

the United States, such as the Common Core State Standards (Common, 2012), the

National Science Education Standards (National Science Teachers Association, 2012),

and STEM learning (Stage & Kinzie, 2009), student achievement level in science has

gained great interest. Many experts cite science literacy as being imperative to full

participation of citizens within society and the global economy (AAAS, 1991; AAAS,

1993; Organisation for Economic Co-operation and Development, 2007) and to

becoming globally competitive citizens who have the knowledge and skills necessary to

make sound scientific decisions in a changing world (AAAS, 1991; Impey et al., 2011;

Lau 2009; Miller, 2007).

The most recent PISA assessment on international science literacy reported a

wide disparity in levels of science literacy among adolescents across the globe

(Organisation for Economic Co-operation and Development, 2007). The PISA

assessment measures student science literacy on a proficiency scale ranging from 1 to 6,

with 1 being the lowest level of science literacy and 6 being the highest level of science

literacy. The assessment measured three science competencies: identification of science

issues, explanation of phenomena using science, and application of scientific evidence

(Organisation for Economic Co-operation and Development, 2007). Results indicated

that only 1.3% of the 15-year old students surveyed across the world reached a level 6

(the highest proficiency level of scientific literacy) (Organisation for Economic Co-

operation and Development, 2007). In addition, 19.2% of students surveyed across the

world scored below a level 2, and 5.2% scored below a level 1 (the lowest proficiency

62

level of scientific literacy) (Organisation for Economic Co-operation and Development,

2007).

Despite different operational definitions of science literacy, six central elements

of science literacy have been identified: “(a) understanding basic science concepts, (b)

understanding nature of science, (c) understanding ethics guiding scientists work, (d)

understanding between science and humanities, and (f) understanding the relationships

and differences between science and technology” (Liu, 2009, p. 302). Among these

central elements, research identified three main types of science literacy: practical, civic,

and cultural (Roberts, 2007). Practical science literacy was defined as the ability to use

science to solve practical problems (Roberts, 2007). Civic science literacy was defined

as a general awareness of science in order to appropriately participate in democratic

processes (Roberts, 2007). Finally, cultural science literacy was defined as the

appreciation of science as a monumental human triumph (Roberts, 2007).

The AAAS, which continues to conduct Project 2061 on the premise of increasing

the scientific literacy of American students, defined science literacy as the knowledge

and habits of mind related to science, mathematics, and technology that individuals must

acquire in order to live interesting, responsible, and productive lives (1993). The result

of Project 2061 was the creation of benchmarks that 90% of American students were

expected to achieve with a mastery of 90% of the prescribed thresholds in the areas of

science, mathematics, and technology, published as Science for All Americans. Science

for All Americans and subsequent supporting publications have striven to balance the

scientific needs of society with the scientific needs of the individual by increasing

63

scientific habits of mind—critical thinking skills that allow a general understanding of

scientific concepts that may be applied to everyday living (AAAS, 1993).

Furthermore, the National Science Foundation, using survey data and data

obtained from PISA assessments, defined science literacy as the knowing of basic

scientific facts and concepts and having a general understanding of how science works

(2004) and presented the Science and Engineering Indicators (NSF, 2010). The Science

and Engineering Indicators reported the quantitative summary information on the scope,

quality, and vitality of the current science environment in an effort to assist in

development of future educational policy related to science (NSF, 2012). While the

Science and Engineering Indicators do not prescribe specific recommendations or

standards for student science achievement, The National Research Council, the National

Science Teachers Association, and the American Association for the Advancement of

Science, and Achieve created the National Science Education Standards to provide

guidance and benchmarks for student science achievement in an effort to increase student

science literacy (National Academy of Sciences, 2013).

The benefits of science literacy are numerous and, in many cases, not easily

measured, but are generally agreed to include the increased ability to make superior

political decisions, the increased ability to reap economic returns, reduction of

misconceptions and superstitions, an increased ability to improve the behavior of the

individual, and the creation of a morally and ethically advanced world (Liu, 2009). Most

experts concur that science literacy will enable citizenship and global competency

necessary in a rapidly changing scientific world (AAAS, 1991; AAAS, 1993; Impey et

al., 2011; NAS, 2013; Roberts, 2007). As such, the concept of science literacy as a whole

64

is difficult to tackle in a single study. Thus, this study focused on a factor that plays an

integral role in science literacy: the misconceptions of science that adolescent students

hold (AAAS, 1993). The identification and reduction of student misconceptions of

science has been identified as paramount to increasing student acquisition of science

knowledge and understanding (Harvard, 2011) as well as student science literacy (AAAS,

1993). Misconceptions may result from the inadequate teaching of scientific concepts,

teacher misconceptions, and preconceived conceptions of science due to student

experience (Harvard, 2011). Student misconceptions may often be difficult to correct and

may produce barriers to increasing student scientific achievement (Burgoon et al., 2011).

Therefore, in order to begin making strides towards realizing national goals towards the

attainment of science literacy, a thorough understanding of practices that affect science

literacy, such as the identification and reduction of student misconceptions of science, is

necessary (AAAS, 1993).

Significance

With the rapidly advancing world of technology that learners experience today

(Black, 2010), understanding the effects of online collaborative learning on students’

sense of community and science literacy is paramount to providing education that meets

the growing needs of today’s learner. Research has shown that collaboration provides

many benefits to learning (Ding & Harskamp, 2011; Miller & Benz, 2008; Parveen &

Batool, 2012; Vaughan et al., 2011; Zhu, 2012). Educational pedagogy has pushed the

implementation of collaborative learning activities to increase student learning and

achievement (Bell et al., 2010).

65

In addition, a shift towards increasing learner opportunities to master scientific

concepts that will allow global competency and participation in society has led to a

renewed interest in student science literacy (AAAS, 1991; Harvard, 2011; Impey et al.,

2011; Miller, 2007; Organisation for Economic Co-operation and Development, 2007).

Since understanding scientific concepts requires higher order thinking that is independent

of rote memorization of facts, a push toward inquiry based learning (Atkin & Black,

2007) that utilizes collaborative activities has been seen recently within the science

classroom (Luft et al., 2008). Research has shown that increased levels of collaborative

instruction have assisted in peer construction of higher order thinking skills, moving

away from rote memorization and simple factual knowledge (Stump et al., 2011).

Discussion in a collaborative setting has also been found to assist in making connections

to prior knowledge and to reduce misconceptions (Webb, et al., 2008), which is important

in identifying student science literacy (Harvard, 2011).

Finally, the social nature of learning is becoming more widely accepted (Lave &

Wenger, 1991) and leads to the importance of understanding sense of community in the

secondary classroom. When students socially engage (such as through collaborative

learning) a learning community is created. Knowledge acquisition is supported and

encouraged when a strong learning community exists, thus fostering excitement and

motivation to learn (Palloff & Pratt, 1999). A strong learning community that produces

an air of enthusiasm for learning may lead to a personal sense of engagement and

empowerment (Palloff & Pratt, 1999), thus increasing student sense of community.

Given the push to implement collaborative activities in the classroom (Bell et al.,

2010), research that examines the effect of student collaboration on student sense of

66

community is both necessary and timely. In considering the challenges of face-to-face

collaborative learning, such as the geographical distance some students now experience,

the encouragement of technology integration in the classroom, and time constraints,

computer-mediated collaboration may be helpful in overcoming those challenges and in

enhancing the learning experience through the use of new teaching tools. However,

weaknesses have also been reported in computer-mediated learning, as the

appropriateness of technology is limited to certain learning tasks, the tools of technology

are inherently structured by the teacher, and the challenges of learning new skills may be

overwhelming for some (Zhao et al., 2004). Furthermore, the implementation of

technology that has shifted how learners communicate and learn (Black, 2010) needs to

be examined more fully in regards to student sense of community. This study, therefore,

sought to fill the current gap in the literature by examining the effect of online

collaboration on both student science literacy and student sense of community among

middle school students.

The next section will examine the methodology of the proposed study. The

research design will be presented and the questions and hypotheses that this study sought

to resolve are defined. The research participants and setting will be detailed. The

measurement instrumentation is examined, including the respective rational for using

each instrument. The research procedures are outlined. Finally, the proposed plan for

analyzing data is stated.

67

CHAPTER THREE: METHODOLOGY

Introduction

The purpose of this study was to determine the effects of online collaborative

learning on middle school students’ science literacy and sense of community. There is a

need for research that examines the effect of different modes of teaching activities, such

as face-to-face and online collaboration, specific to the field of science education as

repeated studies have shown that science literacy amongst adolescents is lacking (AAAS,

1990; Impey et al., 2011; Organisation for Economic Co-operation and Development,

2007). Furthermore, the advantages of collaborative learning (Mäkitalo-Siegl et al.,

2011; Zhu, 2012) and sense of community (Wighting et al., 2009) on student

achievement have been documented. To date, no current studies exist that explore the

effects of face-to-face collaborative learning as compared to online collaborative learning

on science literacy or on sense of community in the adolescent population (Anderson,

2007a; Fraser, 2007). This chapter addresses the methodology proposed for this study,

beginning with the design. The research questions and hypotheses are discussed

followed by a description of the participants and the research setting. As well,

measurement instruments, proposed procedures and data analysis procedures are

presented.

Design

A quasi-experimental, pretest/posttest control group design was used to determine

the effects of online collaborative learning on middle school students’ science literacy

and sense of community. This research design was chosen because the independent

variable was manipulated and a control group was used, but randomization of the sample

68

was not possible due to the educational setting in which the study took place (Gall et al.,

2007; Rovai et al., 2013). As randomization of the sample was not possible, statistical

analysis was conducted to help control for the selection threat to validity (Gall et al.,

2007). A pretest was employed to control for differences in prior science literacy and

community; thus, strengthening the internal validity of the study (Campbell & Stanley,

1963; Gall et al., 2007). Homogenous groups, in terms of gender and socioeconomic

status, were also used.

Questions and Hypotheses

The research questions of this study were as follows.


school students’ misconceptions, an aspect of science literacy, as measured by the







The following were the research hypotheses:





69












learning only.

Alternatively, the following were the null hypotheses:











70






learning only.

Participants

The population of this study included eighth grade middle school students from

five intact general physical science classes at a public middle school in central Virginia.

The study took place during the third nine weeks of the 2012-2013 school year. Two

classes served as the control group and three served as the experimental group. Two

classroom teachers were chosen through recommendations from the county

superintendent and middle school principal based on their quality teaching methods and

history of students’ Virginia Standards of Learning scores. Students in the existing

classes taught by the chosen teacher were used.

Student participants were selected through convenience sampling. Since students

were part of pre-existing groups (classes), the sampling was non-randomized. The

sample was identified from the population through accessibility to the researcher and

through the school district’s willingness to participate. A minimum of 50 students were

chosen for this sample to ensure adequate sample size for the quasi-experimental

pretest/posttest design (Gall et al., 2007) as well as adequate sample size for the statistical

analysis (Cohen, 1988). Statistical texts indicated a minimum sample size of 15 students

71

per group for design (Rovai et al., 2013) and a minimum of 26-64 students per group for

the chosen statistical analysis with a large to medium effect size (Cohen, 1988).

The students in this study completed and returned signed consent and assent

forms. The volunteer rate was 66%. A total of 84 students participated in all portions of

the study. Forty-eight were female, and 36 were male. Through self-reporting, students

were identified as follows: 1.2% Asian/Pacific Subcontinent, 35.3% Black, 2.4%

Hispanic, 1.2% Pacific Islander, 47.1% White, 11.8% two or more races, and 1%

unreported. Specific descriptive information divided by control and experimental group

is shown below (see Table 1).

Table 1

Participant Demographics

Treatment (n = 57 ) Control (n = 27 )

N % N %

Female 32 56.1 16 59.3

Male 25 43.9 11 40.7

African-

American

23 40.4 7 25.9

Caucasian 25 43.9 15 55.6

Hispanic

Asian/Pacific

Subcontinent

Pacific

Islander

Two or more

races

1

1

1

6

1.8

1.8

1.8

10.5

1

0

0

4

3.7

0

0

14.8

72

Setting

Classroom Teachers

Two veteran classroom teachers were used in this study, each with between 9 and

12 years of teaching experience at the middle school level. The classroom teachers held a

professional teaching license in the state of Virginia and were deemed highly qualified

teachers in good standing as measured by professional yearly evaluations by the building

principal and superintendent during implementation of the study. All of the instruction

offered by the classroom teachers was approved by the department chair and aligned with

current district curriculum and current state standards. Thus, instruction of the control

group and experimental group was the same in content and provided to all students in the

same traditional face-to-face format. The medium in which the authentic, collaborative

activities (e.g. traditional face-to-face or online collaborative) were completed served as

the treatment. The authentic, collaborative assignments were identical for both groups

and developed collaboratively by the classroom teachers and researcher. The only

difference was the medium in which the collaborative activities were completed. These

measures taken helped control for instrumentation threats and construct threats to internal

validity.

Overview of the School

The school was an accredited, public middle school in South-Central Virginia and

will be referred to by the pseudonym Central Virginia Middle School (CVMS). The

school was part of a rural school system that serves approximately 4500 students. At the

time of this study, 4453 students were enrolled during the 2012-2013 school year with a

ratio of 49% female and 51% male. The demographics of the school system include

73

5.2% Hispanic, 1% American Indian/Alaska Native, <1% Asian, 34.7% Black/African

American, <1% Native Hawaiian/Other Pacific Islander, 54.8% White, and 4.3% two or

more races.

Science Classroom

The study took place over a nine week grading period in the eighth grade physical

science class, a course required by the school system and the Virginia Department of

Education. As such, it is a Virginia Standards of Learning (SOL) course. Students

enrolled in physical science are required to complete and successfully pass the Virginia

SOL for physical science at the end of the school year. At the time of this study, a score

of 400/600 is required to pass.

The physical science curriculum served as the framework for this study. Topics

covered in physical science include scientific investigation, force, motion, energy, matter,

life processes, living systems, interrelationships in earth and space systems, earth

patterns, cycles, and earth resources (Virginia, 2010). These topics were covered in

accordance with the Virginia SOL standards. According to the school system pacing

guide, the specific topics covered during the study implementation period included

thermal energy and heat, work and machines, states of matter, matter change, and atomic

structure. These topics coincide with the following Virginia SOL standards: PS.2, PS. 4,

PS. 5, PS. 6, PS. 7, and PS. 10 (Virginia, 2010) and the following National Science

Education Standards: Physical Science 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11 (National

Academy of Sciences, 1996).

The curriculum was held constant across all classrooms for this study. The

control group engaged in collaborative activities completed in a traditional, or face-to-

74

face, manner. The experimental group engaged in collaborative activities facilitated by

computer-mediated tools, specifically the use of Edmodo. Further information regarding

the specific settings is provided in the next section.

Collaboration Settings

Computer-mediated collaborative activities for the experimental group were

conducted through use of Edmodo educational platform. Edmodo is an educational

learning platform that allows social networking for teacher and student connection and

collaboration (Edmodo, 2012). Students selected a username and passcode and were

provided with an access code to gain access to specific course information. Edmodo

allowed students to engage in group discussion through creation of posts and allowed the

teacher to upload documents and multimedia. Use of the site is free of charge and

currently serves approximately 15 million teachers and students worldwide (Edmodo,

2012). A screenshot can be seen in Figure 1.

75

Figure 1. A screenshot of the Edmodo homepage is provided here. Edmodo, an online

educational platform, has similarities to common social networking sites and allows

collaboration among peers, teachers, and parents. (Retrieved from

http://www.edmodo.com/home#/.)

In the experimental group, students completed the collaborative activities in a

combination synchronous and an asynchronous manner. Students participated in

collaborative activities through the use of the Edmodo educational platform with

members of a small group that was created at the discretion of the classroom teacher. In

addition, students participated in collaborative activities through the use of the Edmodo

educational platform with members outside of their small group and outside of their

immediate class. Thus, students were able to collaborate through discussion, inquiry, and

reflection with the entire experimental group sample pool of students and were not

limited by the physical walls of the classroom. Students were monitored by the

76

classroom teacher as the classroom teacher ensured that all activities were completed

through observations and marking completed activities in the gradebook. The teacher

encouraged peer-to-peer collaboration with classmates through verbal promotion to

consider alternative student viewpoints and ideas. The teacher did not provide immediate

feedback and re-direction via Edmodo; instead, due to the nature of the asynchronous

learning environment, rather concepts were discussed during instructional class time.

In the control group, students completed the collaborative activities using

tradition face-to-face collaborative methods. Students were divided into small groups at

the discretion of the classroom teacher using pre-existing, intact groups, and were

encouraged through verbal means to complete activities collaboratively, engaging in

discussion, inquiry, and reflection. Students were monitored by the classroom teacher,

who provided immediate feedback and re-direction when appropriate.

The arrangement of desks was similar for each classroom, with desks being

arranged in rows facing the front of the classroom and the teacher (see Figure 2).

Students in the control group were allowed to move their desks for collaborative

activities to facilitate communication. One classroom teacher was available for each

group. Although due to the nature of the settings, the control group teacher provided

immediate feedback while the experimental group teacher provided delayed feedback

through whole-class discussion after the conclusion of the collaborative activities.

77

Figure 2. Approximate arrangements of student desks and chairs arranged in rows facing

the front of the classroom and the teacher in the physical classroom environment.

Explanation of Collaborative Activities

The collaborative activities in this study employed group discussion and sharing

of ideas in order to build upon individual members’ histories and experiences. Both the

experimental and control groups completed equivalent activities—in many cases, the

activities consisted of the same figures and questions. The activities in this study were

selected to be convenient for normal classroom instruction; that is, the activities chosen

for inclusion in this study were meant to be completed in fifteen minutes or less and were

intended to reiterate teacher-presented material through discussion and brief reflection.

A minimum of two activities were completed per week for a total of nine weeks

(one grading period). Permission to use, distribute, and copy materials was provided by

Teacher Desk

Student Desks and Chairs

78

Pearson, and a letter of permission is included in Appendix M. Table 2 shows the

sequence of activities for both the experimental and control groups and is found below.

In addition, examples of each activity are provided in Appendix N.

Table 2

Description of Sequence, Content, and Corresponding VA Standards of Learning of

Collaborative Activities

Topics Covered Corresponding VA

Standard of Learning

Activity 1 The Meaning of Work; Calculating Work PS.6, PS.10

Activity 2 Identifying Resistance and Effort PS.6, PS.10

Activity 3 What is a Machine? PS.6, PS.10

Activity 4 Mechanical Advantage and Efficiency PS.6, PS.10

Activity 5 Structure of an Atom PS.3, PS.4

Activity 6 The Role of Electrons PS.3, PS.4

Activity 7 The Periodic Table; Organizing the Elements PS.3, PS.4

Activity 8 Why the Periodic Table Works PS.3, PS.4

Activity 9 Metals and Alloys PS.4

Activity 10 Nonmetals and Metalloids PS.4

Activity 11 Families of Nonmetals PS.4

Activity 12 Particles of Matter; States of Matter PS.2, PS.5

Activity 13 Describing Matter PS.2, PS.5

Activity 14 Changes in State PS.2, PS.5

Activity 15 Changes of State; Thermal Energy PS.2, PS.5,

PS.6

Activity 16 Thermal Expansion PS.6

Activity 17 Heat Engines PS.6, PS.7

Activity 18 Refrigerators PS.7

79

All activities were correlated with the Virginia Standards of Learning as well as the

National Science Standards. National Science Standards are listed and numbered in

Table 3. Virginia Standards of Learning are listed and numbered in Table 4.

Table 3

National Science Education Standards as published by the National Academy of Sciences

Standard Description of Standard

1 A substance has characteristic properties, such as density, a boiling point, and solubility, all

of which are independent of the amount of the sample. A mixture of substances often can

be separated into the original substances using one or more of the characteristic properties.

2 Substances reach chemically in characteristic ways with other substances to form new

substances (compounds) with different characteristic properties. In chemical reactions, the

total mass is conserved. Substances often are placed in categories or groups if they react in

similar ways; metals is an example of such a group.

3 Chemical elements do not break down during normal laboratory reactions involving such

treatments as heating, exposure to electric current, or reaction with acids. There are more

than 100 known elements that combine in a multitude of ways to produce compounds,

which account for the living and nonliving substances that we encounter.

4 The motion of an object can be described by its position, direction of motion, and speed.

That motion can be measured and represented on a graph.

5 If more than one force acts on an object along a straight line, then the forces will reinforce

or cancel one another, depending on their direction and magnitude. Unbalanced forces will

cause changes in the speed or direction of an object’s motion.

6 Energy is a property of many substances and is associated with heat, light, electricity,

mechanical motion, sound, nuclei, and the nature of a chemical. Energy is transferred in

many ways.

7 Heat moves in predictable ways, flowing from warmer objects to cooler ones, until both

reach the same temperature.

8 Light interacts with matter by transmission (including refraction), absorption, or scattering

(including reflection). To see an object, light from that object—emitted by or scattered

from it—must enter the eye.

9 Electrical circuits provide a means of transferring electrical energy when heat, light, sound,

and chemical changes are produced.

10 In most chemical and nuclear reactions, energy is transferred into or out of a system. Heat,

light, mechanical motion, or electricity might all be involved in such transfers.

11 The sun is a major source of energy for changes on earth’s surface. The sun loses energy

by emitting light. A tiny fraction of that light reaches earth, transferring energy from the

sun to the earth. The sun’s energy arrives as light with a range of wavelengths, consisting

of visible light, infrared, and ultraviolet radiation.

(National Academy of Sciences, 1996, p. 165-166)

80

Table 4

Virginia Standards of Learning for Physical Science as listed by the Virginia Department

of Education


PS.1 The student will demonstrate an understanding of scientific reasoning, logic, and the nature of

science by planning and conducting investigations in which

a) chemicals and equipment are used safely;

b) length, mass, volume, density, temperature, weight, and force are accurately measured;

c) conversions are made among metric units, applying appropriate prefixes;

d) triple beam and electronic balances, thermometers, metric rulers, graduated cylinders,

probeware, and spring scales are used to gather data;

e) numbers are expressed in scientific notation where appropriate;

f) independent and dependent variables, constants, controls, and repeated trials are

identified;

g) data tables showing the independent and dependent variables, derived quantities, and the

number of trials are constructed and interpreted;

h) data tables for descriptive statistics showing specific measures of central tendency, the

range of the data set, and the number of repeated trials are constructed and interpreted;

i) frequency distributions, scatterplots, line plots, and histograms are constructed and

interpreted;

j) valid conclusions are made after analyzing data;

k) research methods are used to investigate practical problems and questions;

l) experimental results are presented in appropriate written form;

m) models and simulations are constructed and used to illustrate and explain phenomena;

and

n) current applications of physical science concepts are used.

PS.2 The student will investigate and understand the nature of matter. Key concepts include

a) the particle theory of matter;

b) elements, compounds, mixtures, acids, bases, and salts;

c) solids, liquids, and gases;

d) physical properties;

e) chemical properties; and

f) characteristics of types of matter based on physical and chemical properties.

PS.3 The student will investigate and understand the modern and historical models of atomic

structure. Key concepts include

a) the contributions of Dalton, Thomson, Rutherford, and Bohr in understanding the atom;

and

b) the modern model of atomic structure.

PS.4 The student will investigate and understand the organization and use of the periodic table of

elements to obtain information. Key concepts include

a) symbols, atomic numbers, atomic mass, chemical families (groups), and periods;

b) classification of elements as metals, metalloids, and nonmetals; and

c) formation of compounds through ionic and covalent bonding.

PS.5 The student will investigate and understand changes in matter and the relationship of these

changes to the Law of Conservation of Matter and Energy. Key concepts include

a) physical changes;

b) chemical changes; and

c) nuclear reactions.

PS.6 The student will investigate and understand forms of energy and how energy is

transferred and transformed. Key concepts include

a) potential and kinetic energy; and

81


b) mechanical, chemical, electrical, thermal, radiant, and nuclear energy.

PS.7 The student will investigate and understand temperature scales, heat, and thermal energy

transfer. Key concepts include

a) Celsius and Kelvin temperature scales and absolute zero;

b) phase change, freezing point, melting point, boiling point, vaporization, and

condensation;

c) conduction, convection, and radiation; and

d) applications of thermal energy transfer.

PS.8 The student will investigate and understand the characteristics of sound waves. Key concepts

include

a) wavelength, frequency, speed, amplitude, rarefaction, and compression;

b) resonance;

c) the nature of compression waves; and

d) technological applications of sound.

PS.9 The student will investigate and understand the characteristics of transverse waves. Key

concepts include

a) wavelength, frequency, speed, amplitude, crest, and trough;

b) the wave behavior of light;

c) images formed by lenses and mirrors;

d) the electromagnetic spectrum; and

e) technological applications of light.

PS.10 The student will investigate and understand the scientific principles of work, force, and

motion. Key concepts include

a) speed, velocity, and acceleration;

b) Newton’s laws of motion;

c) work, force, mechanical advantage, efficiency, and power; and

d) technological applications of work, force, and motion.

PS.11 The student will investigate and understand basic principles of electricity and magnetism.

Key concepts include

a) static electricity, current electricity, and circuits;

b) relationship between a magnetic field and an electric current;

c) electromagnets, motors, and generators and their uses; and

d) conductors, semiconductors, and insulators.

(Virginia Department of Education, 2010, p. 6-8)

Furthermore, activities were correlated with the National Science Standards that

the MOSART Physical Science Assessment specifically tested for as shown in Table 5.

82

Table 5

MOSART Physical Assessment Item Correlations with National Science Standards and

Virginia SOL Standards

Item # (Form 921) Item # (Form 922) National Science Virginia SOL

Standard Standard

1 18 1 PS.2, PS.7

2 13 3 PS.2

3 16 7 PS.7

4 7 9 PS.11

5 9 11 PS.9

6 17 2 PS.4, PS.5

7 14 4 PS.10

8 20 6 PS.6

9 4 8 PS.9

10 19 10 PS.6

11 5 11 PS.9

12 8 7 PS.7

13 15 2 PS.4, PS.5

14 12 4 PS.10

15 10 3 PS.2

16 6 5 PS.10

17 2 8 PS.9

18 3 9 PS.11

19 11 1 PS.2, PS.7

20 1 1 PS.2, PS.7

Instrumentation

Student science literacy was measured using the Misconceptions-Oriented

Standards-Based Assessment Resources for Teachers (MOSART) assessment (Harvard,

2011), which is designed to measure misconceptions as an aspect of science literacy (H.

Coyle, personal communication, January 24, 2013). The Harvard-Smithsonian Center for

Astrophysics, with funding from the National Science Foundation (NSF), completed

development of the first MOSART assessments in 2001 (Harvard-Smithsonian Center for

Astrophysics, 2012; Smith, n.d.). The purpose of the project was to develop a diagnostic

83

tool to assist science instructors at the elementary, middle, and high school levels in

identifying student misconceptions of science concepts and to evaluate the extent of

mastery of national science standards and AAAS Benchmarks in Physical Science and

Earth and Space Science (Center for School Reform at TERC, 2012). The scope of the

MOSART assessments has since extended to Astronomy, Life Science, Chemistry, and

Physics. The resulting MOSART assessments are divided into appropriate grade level

(K-4, 5-8, and 9-12) assessments and further subdivided by subject area

(Astronomy/Space Science, Earth Science, Life Science, Physical Science, Chemistry,

and Physics) (Harvard, 2011).

Each MOSART assessment contains multiple choice questions for each of the

correlating K-12 National Science Standards, with five student answer choice options for

each question. Assessment questions were chosen from a test bank with over 2000

questions compiled over ten years. These questions were created by research experts in

the science field in tandem with a development team and then were reviewed by one

literacy expert to ensure readability and grade appropriateness (Harvard, 2011; Sadler et

al., 2010; Smith, n.d.). Pilot versions of the tests were then created and administered to

elementary, middle, and high school students. Results were analyzed and field tests of

the revised assessment instruments were conducted. The KR-20 score of 0.85, a measure

of high test item reliability, was found for grades 9-12 (Sadler et al., 2010). Cronbach’s

alpha ranged from 0.7-0.9 for all tests, thus indicating internal reliability (Smith, n.d.).

Validity of the MOSART assessments was ensured through a scientific review process,

item fit review, and uni-dimensionality review (Smith, n.d.). For this study, Cronbach’s

α = .98, indicating increased internal reliability (Tabachnick & Fidell, 2013).

84

Access to the MOSART assessments requires completion of online tutorials

provided on the MOSART Self-Service website. These tutorials include information on

the definition of misconceptions, how classroom teachers may identify student

misconceptions of science, the intent of the MOSART assessments, scoring of the

MOSART assessments, analyzing data obtained from MOSART assessments, and how to

use data to influence the practice of teaching science in the classroom. Upon completion

of the tutorials, the MOSART assessments are available by request in digital format. An

answer key is provided for each publicly released MOSART assessment, which is

typically scored by the classroom teacher (Harvard, 2011). Assessment follows a

distractor analysis approach, meaning the classroom teacher scores the assessments

through counting the number of students who choose each response option (Harvard,

2011). It is possible that allowing the classroom teacher to score the assessment while

having access to the test forms may introduce bias. However, the intent of the MOSART

assessment is to assist the classroom teacher in identifying student misconceptions in

order to make sound improvements in science teaching practice, thus negating any

benefit of “teaching to the test”. The classroom teacher then determines areas of

misconceptions by identifying questions where one incorrect response is more prevalent

than other incorrect responses. Scores may be expressed as percentages correct with a

range from 0%-100%, with approximately 25% of the assessment questions expected to

be easy, 25% difficult, and 50% moderate (Harvard, 2011). Additionally, each

MOSART assessment consists of two parallel tests to promote use in a pretest/posttest

design (Harvard, 2011). The parallel tests have identical content and psychometric

characteristics.

85

The MOSART assessments are publicly available for classroom and research use

once online tutorial completion requirements have been met. However, additional

permission to use the MOSART assessments as part of this study was granted by Mr. Hal

Coyle, Manager of MOSART Projects through email correspondence (Appendix A). The

MOSART assessments were provided to students as a pencil and paper test. A printed

copy of the MOSART assessment is found in Appendix C. Student answers were marked

on a bubble-sheet that is compatible with the Reports Online Systems (ROS) ®. The

ROS® system provided a means for aggregation of results; results were then imported

into the SPSS program for data analysis.

To measure student sense of community, the Classroom Community Scale was

used in this study (Rovai, 2002a). The Classroom Community Scale was developed from

elements of classroom community identified through an extensive review of the literature

(Rovai, 2002a); elements consisted of “feelings of connectedness, cohesion, spirit, trust,

and interdependence among members” (Rovai, 2002a, p. 201). A set of 20 questions was

created to address the identified elements of sense of community. Additional questions

were later added in order to address community issues specific to either the traditional or

virtual classroom, thus the final Classroom Community Scale is appropriate for use in

both traditional face-to-face classrooms as well as virtual or online classrooms (Rovai,

2002a). The questions were then rated by a panel of educational psychology experts to

determine relevancy and identify issues of factor loading (correlation between factors and

variables), resulting in the deletion of non-relevant items. The final Classroom

Community Scale was then created with a total of 20 questions with a range of responses

using a 5-point Likert-type scale.

86

The Classroom Community Scale was empirically tested for reliability with a

Cronbach’s coefficient α of .93 (Rovai, 2002a). Validity of the Classroom Community

Scale was ensured through the ratings of three university professors who taught

educational psychology, as well as grounding each item in the professional literature

(Rovai, 2002a). Furthermore, the readability and ease of understanding of the Classroom

Community Scale were made certain through a Flesch Reading Ease score of 68.4 and

Flesch-Kincaid grade level score of 6.6 (Rovai, 2002a).

Finally, two subscales of the Classroom Community Scale have been identified:

connectedness and learning (Rovai, 2002a). Internal consistency was estimated for each

subscale, with a Cronbach’s α of .92 for the connectedness subscale and a Cronbach’s α

of .87 for the learning subscale. Thus, both subscales showed reliability (Rovai, 2002a).

The Classroom Community Scale is appropriate for use on adult populations (Rovai,

2002a) and adolescent students (Rovai, Wighting, & Lucking, 2004; Wighting et al.,

2009). For this particular study, reliability for overall sense of community was calculated

as Cronbach’s α = .80 for the pre-test survey, indicating high reliability (Rovai et al.,

2013). Reliability for the subscale connectedness was calculated as Cronbach’s α = .75

and for the subscale learning as Cronbach’s α = .68, indicating moderate to high

reliability among the subscales. Reliability for overall sense of community was

calculated as Cronbach’s α = .80 for the post-test survey, indicating high reliability.

Reliability for the subscale connectedness was calculated as Cronbach’s α = .77 and for

the subscale learning as Cronbach’s α = .65, indicating moderate to high reliability

among the subscales (Rovai et al., 2013).

87

Scoring of the Classroom Community Scale is completed by instructing students

to mark the area on the Likert-type scale that most appropriately describes their feelings

about the item; then the researcher computing scores by adding points that are pre-

assigned to each of the items, with the most favorable choice being assigned a value of 4

and the least favorable choice being assigned a value of 0 (Rovai, 2002b). Possible

scores may range from 0 to 80. A higher score reflects a strong sense of community

while a lower score reflects a low sense of community (Rovai, 2002b). Scores for each

of the subscales may range from 0 to 40.

Permission to use the Classroom Community Scale is provided, as “researchers

may use this instrument [the Classroom Community Scale] for studies they conduct

provided they give proper attribution by citing this article” (Rovai, 2002a, p. 202). In

addition, specific permission was granted by Rovai through email correspondence

(Appendix B). A print version of the Classroom Community Scale is found in Appendix

D. The Classroom Community Scale, including additional questions to gather data on

gender and ethnicity was completed as an online survey utilizing the SurveyMonkey®

online survey program. On overview of the testing instruments is provided in Table 6.

88

Table 6

Description of Measurement Instruments

MOSART Classroom

Community Scale

Construct Measured Misconceptions of science Sense of community

Format of Assessment Multiple choice Survey; Likert-type

scale

Reliability Cronbach’s α = 0.7-0.9 Cronbach’s α = 0.93

Validity Reported as verified through Reported as

item fit, uni-dimensionality possessing high

content and construct

validity

Score Range 0-100 (percentage) 0-40 (points) per

subscale

Subscales None Learning and

Connectedness

Procedures

After submitting the dissertation proposal packet and gaining IRB approval,

execution of the study began. Consent and assent forms (Appendices E & F,

respectively) were provided to all potential students and collected by the classroom

teachers; those students included in the study had signed students informed consent an

assent forms. The researcher and classroom teachers met on two days to choose

collaborative activities to provide to the students for this study. The two classroom

teachers participating in administration of the testing materials and instruction were

provided with MOSART training prior to implementation of the study through the use of

four online tutorials publicly available at the MOSART Self-Service Site (Harvard,

2011). Instructions for how to complete the MOSART training were provided to the

89

classroom teachers through Word document and email correspondence (Appendix I). In

addition, approximately one hour of face-to-face training was provided to the

experimental group’s classroom teacher by the researcher on the use of the Edmodo

educational platform.

At the beginning of the study, the classroom teachers instructed students in both

the experimental and control groups to complete the pretest materials consisting of the

MOSART assessment and the Classroom Community Scale survey. The classroom

teachers were provided a script to use in administration of both the MOSART assessment

and Classroom Community Scale survey for both groups (Appendices G & H,

respectively); this was used in both the pretesting and posttesting. The completed tests

and surveys were given to the researcher for scoring and analysis. The classroom

teachers provided normal in-class instruction to both the experimental and control groups.

The teacher assigned to the control group provided students with assignments that they

were instructed to complete collaboratively in a traditional face-to-face manner. The

teacher assigned to the experimental group provided students with the same assignments

as the control group. However, the experimental group was given the instruction to

complete the assignments in class collaboratively through the use of Edmodo.

Collaborative assignments were given to both groups at a minimum of two times weekly

at the discretion of the classroom teachers and on days determined by the classroom

teachers. Completed assignments for both the experimental and control group were

marked in the teachers’ grade book by the classroom teachers to ensure minimum

participation of two completed assignments for each participant per week. This

continued for nine weeks (one grading period). Fidelity of treatment was ensured

90

through equivalent classroom instruction provided by the classroom teachers to all

students to the best of their ability as measured by review of teacher lesson plans by the

researcher.

At the end of the nine-week grading period, the classroom teachers instructed

students in both groups to complete the posttest materials consisting of the MOSART

assessment and the Classroom Community Scale survey. The completed tests and

surveys were given to the researcher for scoring and analysis. No incentive was provided

for student students as the completion of the MOSART assessments were considered part

of the normal curriculum. Students whose consent forms were not returned to the

classroom teacher still participated since the MOSART assessment was considered part

of the established curriculum; however, their data was not included in the final data

analysis. Furthermore, students whose consent forms were not returned to the classroom

teacher did not participate in completion of the Classroom Community Scale survey as

the survey was not considered part of the normal curriculum.

Data Analysis

Research Question One

One-way analysis of covariance (ANCOVA) was used to examine the null

hypothesis: There is no statistically significant difference in middle school students’




The type of medium for collaboration served as the independent variable for both

analyses. Misconceptions, an aspect of science literacy measured via the MOSART

91

served as the dependent variables. No subscales have been reported in regards to the

MOSART assessment within the literature. The MOSART pretest served as the

covariate. The ANCOVA analysis was deemed most appropriate when one or more

covariates exist and are used to adjust for differences in pre-test scores (Tabachnick &

Fidell, 2013; Rovai et al., 2013).

Independent t-tests for the pre-test scores were conducted to determine if there

was a significant difference in MOSART pretest assessment scores based on group

assignment prior to interventions. A statistically significant difference was found on the

pretest suggesting pre-existing differences between groups in science misconceptions;

thus, an ANCOVA analysis was most appropriate in order to examine posttest differences

while controlling the pretest. A scatterplot was also inspected which revealed a

correlation, although weak, between variables. This further confirmed the choice of the

ANCOVA.

Prior to the analysis, assumption testing was completed. Normality was tested by

examination of histograms, which showed a distribution that indicated normality among

the control group pre-test and posttest MOSART data and negative skewness among the

experimental group pre-test and posttest data. Furthermore, Shapiro-Wilk and

Kolmogorov Smirnov were used to verify the assumptions of normality. Linearity, the

assumption that the rate of change between the scores of two variables is constant, was

assessed using a scatterplot (Rovai et al., 2013). The assumption of linearity was met as

an approximate straight line existed between the variables. Homogeneity of variance,

also known as error variance, was tested for through Levene’s test. It assessed the null

hypothesis that the variance of the dependent variables were equal across groups

92

(Levene, 1960; Rovai et al., 2013). Analysis for this indicated that the assumption of

homogeneity of variance was tenable. Homogeneity of regression slopes, which is used

to examine whether the slopes of the regression lines are the same for each group (Rovai

et al., 2013), was examined since covariates were found. The assumption of

homogeneity of regression slopes was tenable. Additional assumption testing was

conducted and is discussed in Chapter 4.

The overall F-test was examined for the ANCOVA (Rovai, et al., 2013). Effect

size, the practical significance of the magnitude of the treatment, was calculated as eta

square (ƞ2) (Rovai, et al., 2013) and interpreted using Cohen’s d (Cohen, 1988; Rovai, et

al., 2013). The 0.05 significance level was used to determine whether the null hypotheses

were rejected (Rovai et al., 2013). A significance level of 0.05 is generally accepted

within social science research and indicates the probability of making a Type I error or

falsely rejecting the null hypothesis (Rovai et al., 2013). An overview of the test of

statistical analyses is provided in Table 7.

Research Question Two

One-way multivariate analyses of variance (MANOVA) and individual one-way

analyses of variance (ANOVA) were used to analyze null hypotheses 2 through 4. Since

the subscales of the CCS were analyzed, and the subscales were found to be correlated

during assumption in this study, a MANOVA was appropriate as it evaluates the

significance of group differences between two or more groups when there are correlated

dependent variables (Tabachnick & Fidell, 2013). Additionally, there was not a need to

control covariates. An independent t-test for the pre-test scores was conducted to

determine if there was a significant difference in scores based on group assignment prior

93

to interventions. No differences were found indicating that there was not a need to

control for pre-existing difference in community. Prior to conducting the analysis,

assumption testing was completed and is discussed in Chapter 4.

While MANOVA analysis is typically robust in regards to normality (Tabachnick

& Fidell, 2013), normality and multivariate normality were tested for by completing

histograms (Gall et al., 2007). Inspection of the histograms showed a distribution that

indicated normality among the control group learning and connectedness and

experimental group learning posttest data. However, inspection of the histograms

showed that the experimental group connectedness data was slightly positively skewed.

Furthermore, Shapiro-Wilk and Kolmogorov Smirnov were used to verify that

assumptions of normality were not violated. Multivariate normality was examined

through Mahalanobis distance, a measure of the multivariate outliers (Tabachnick &

Fidell, 2013). Mahalanobis distance creates points at the intersection of the means of all

variables, thus creating a swarm of points around the centroid which also indicates

multivariate normality (Tabachnick & Fidell, 2013). Points that lie outside of the swarm

are considered outliers and are generally removed from data. For this study, one extreme

outlier was found and removed.

Linearity, the assumption that the rate of change between the scores of two

variables is constant, was assessed using a scatterplot (Rovai et al., 2013). The

assumption of linearity would be met if an approximate straight line existed between the

variables, which was indicated through inspection of data for this study. Furthermore, a

matrix of scatterplots was examined to ensure the assumptions of multicollinearity and

singularity were upheld.

94

Homogeneity of covariance and variance, also known as error variance, was

tested for through Box’s M test and Levene’s respectively (Rovai, et al., 2013). A

significance level of p < .001 indicates a violation exists (Tabachnick & Fidell, 2013).

Box’s M tests MANOVA’s assumption of homogeneity of covariance matrices using the

F distribution. In order for the assumption of homogeneity of covariance matrices to be

upheld, the probability value should be greater than 0.05, meaning that M is found to be

not significant (Rovai et al., 2013). For this study, the assumption of homogeneity of

covariance was not violated. Levene’s test assessed the null hypothesis that the variance

of the dependent variables was equal across groups (Rovai et al., 2013). Levene’s test is

generally accepted to be robust when departures from normality are seen (Rovai et al.,

2013). Levene’s test was not significant in this study.

The overall F-test was examined for the MANOVA and individual ANOVA

analyses (Rovai, et al., 2013). Effect size, the practical significance of the magnitude of

the treatment, was calculated as eta square (ƞ2) (Rovai, et al., 2013) and interpreted using

Cohen’s d (Cohen, 1988; Rovai, et al., 2013). The 0.05 significance level was used to

determine whether the null hypothesis for the MANOVA was rejected (Rovai et al.,

2013). A significance level of 0.05 is generally accepted within social science research

and indicates the probability of making a Type I error or falsely rejecting the null

hypothesis (Rovai et al., 2013). A more stringent alpha, Bonferroni correction, was set to

control for Type I error (Rovai et al., 2013). Bonferroni was calculated as α = .025

(.05/2).

Statistical convention requires that the sample size for MANOVA analysis exceed

the number of dependent variables in each of the cells (Tabachnick & Fidell, 2013),

95

which was met for this study. The number of students needed for each group was 15 for

the experimental design (Rovai et al., 2013). Cohen (1988) suggests that the number of

students needed for each group to be 26 for the MANOVA statistical analysis. This study

aimed to use approximately 50 students in each group. An overview of the test of

statistical analyses is provided in Table 7.

Table 7

Organization of Statistical Analysis of Data

Statistical test Purpose

ANCOVA Analysis of hypothesis for research question one

MANOVA Analysis of the hypothesis two for research question

two

ANOVA Analysis of hypothesis three and four for research

question two

Independent t Tests Test for significant differences in scores based on

group assignment prior to interventions

Scatterplot and correlation Correlation, linearity, multicollinearity, singularity

coefficient

Histograms Normality

Shapiro-Wilk Normality

Kolmogorov-Smirnov Normality

Mahalanobis Distance Normality

Box plots Normality

Levene’s Test Homogeneity of variance

Scatterplot Homogeneity of regression slopes

Effect Size Practical significance of the magnitude of the treatment

(calculated as eta square; interpreted using Cohen’s d)

Box’s M Homogeneity of covariance

96

The aim of this quasi experimental pre-test/posttest control group design

experiment was to determine the effects of online collaboration on middle school

students’ science literacy and sense of community among a representative sample of

middle school physical science students in a rural public school system in South-Central

Virginia. In the next section, the findings of the research study are presented. The results

of each hypothesis tested will be discussed.

97

CHAPTER FOUR: FINDINGS

Restatement of the Purpose

The purpose of the study was to investigate the effect of online collaborative

learning on middle school student science literacy and sense of community. Students

were eighth grade students enrolled in pre-existing general education physical science

classes at an accredited public middle school in South-Central Virginia. Given the

current push to increase adolescent science literacy and to improve understanding of best

practices in the science classroom, this study was timely. In addition, in light of current

efforts to increase technology implementation in the classroom and the move towards

addition of distance education courses in the public school systems, this study was

timely. This study contributed to the body of knowledge in regards to the effect online

collaboration may have on student science literacy with a specific focus on

misconceptions. This study also provided relevant literature that investigated the effect

of online collaboration on adolescent sense of community, with particular emphasis on

the science classroom.

Research Questions and Hypotheses

The following research questions were investigated:


school students’ misconceptions, aspect of science literacy, as measured by the





98



The following were the corresponding research hypotheses:
















learning only.

Alternatively, the following null hypotheses were tested:



99














learning only.

Demographics

A total of 84 students were part of this study, all of whom were eighth grade

physical science students enrolled in an accredited public middle school in central

Virginia—Central Virginia Middle School (CVMS). All students were existing members

of pre-existing general education physical science classes. The regular classroom

teachers provided classroom instruction.

Of the 84 students, 48 were female and 36 were male. None of the students had a

formal educational plan on file, which indicated that none of the students were students

with disabilities. Within the experimental group, n = 57 and within the control group, n =

100

27. Specific descriptive data detailing the race and gender of each of the participant

students within each group is presented in Chapter 3. Information regarding

socioeconomic status was not collected as part of this study due to the likelihood of false

self-reporting due to the age of the students involved and the chance of students being

unaware of an accurate report of family income.


Research question one was as follows: Is there a statistically significant

difference in middle school students’ misconceptions, an aspect of science literacy, as

measured by the MOSART testing instrument when participating in online collaborative

learning as compared to students who participate in traditional collaborative learning

only? An analysis of covariance (ANCOVA) examines whether the means of groups are

statistically different from one another while controlling for the effects of a potentially

confounding variables (Rovai et al., 2013). An ANCOVA analysis was used to analyze

the first null hypothesis. Ho1: There is no statistically significant difference in middle

school students’ misconceptions of science as measured by the MOSART testing

instrument when participating in online collaborative learning as compared to students

who participate in traditional collaborative learning only, while controlling for student

prior knowledge. Assumption testing was conducted prior to running the analysis and is

explained in the next section.

Assumption Testing

An independent t-test was first conducted to ensure that no statistically significant

difference existed among the means of the control and experimental group’s pretest

scores on the MOSART (Rovai et al., 2013). Statistically significant difference in scores

101

for the control group (M = 4.81, SD = 1.80) and the experimental group (M = 6.17, SD =

2.31); t (88) = 2.86, p = .005 were found. The magnitude of the differences in the means

(mean difference = 1.36, 95% CI: -2.31 to -.42) was moderate to large (eta square = .09);

thus, analysis of covariance (ANCOVA) was used to control for preexisting differences

(Tabachnick & Fidell, 2013).

Upon inspection of a scatterplot of MOSART pre-test and posttest data, a weak

correlation was found (see Figure 3). Therefore, controlling for the covariate was further

deemed appropriate (Rovai et al., 2013).

Figure 3. Scatterplot of MOSART pre-test and posttest data showing a weak correlation.

Normality

Normality was tested for through construction of histograms. Histograms showed

a normal distribution in the control group pre-test and posttest MOSART data and

102

negatively skewed distribution among the experimental group pre-test and posttest data

(Tabachnick & Fidell, 2013) (see Figure 4).

103

Figure 4. Histograms for normality testing of research question one.

Normality for the MOSART data was also tested for through use of Shapiro-Wilk

and Kolmogorov-Smirnov (Tabachnick & Fidell, 2013). Since the control group

contained less than 50 participants, results of Shapiro-Wilk (Tabachnick & Fidell, 2013)

were used to determine that the control group did not violate assumptions of normality (p

= .351 which was greater than α = .05). Since the experimental group contained more

than 50 participants, results of Kolmogorov-Smirnoff (Tabachnick & Fidell, 2013) were

104

used to determine that the experimental group did violate assumptions of normality (p =

.001 which was less than α = .05). However, the ANCOVA is still considered robust

when the number of participants exceeds 20 (Tabachnick & Fidell, 2013). Additionally,

inspection of boxplots indicated that no violation of assumptions of extreme outliers for

the MOSART data; thus, the assumption of no extreme outliers was tenable.

Linearity

Linearity was examined through inspection of a scatterplot of MOSART pre-test

and posttest data (see Figure 3). The assumption of linearity was not violated; the

relationship between the variables was linear.

Variance

The assumption of homogeneity of variance for the MOSART data was examined

with Levene’s test. Levene’s test assesses the null hypothesis that the variance of the

dependent variables were equal across groups (Rovai et al., 2013). Levene’s test is

generally accepted to be robust when departures from normality are seen (Rovai et al.,

2013). Levene’s test was not significant, and; thus, the assumption of homogeneity of

variance was tenable for the MOSART posttest data, F(1, 88) = 2.01, p = .16 .

Homogeneity of regression slopes

Homogeneity of regression slopes was tested for the MOSART data.

Homogeneity of regression slopes is used to examine whether the slopes of the regression

lines are the same for each group (Rovai et al., 2013). When this assumption is violated,

the probability of making Type I errors by use of the covariate procedure increases

(Rovai et al., 2013). A two-way between-groups ANOVA was conducted to analyze

homogeneity of regression slopes. The results indicated F(1, 86) = 2.7, p = .10. Since p

105

= .10 is greater than α = .05, the assumption of homogeneity of regression slopes was not

violated.

Reliability of Covariates

The reliability measure of the MOSART assessment was found to be a

Cronbach’s α = .98, indicating appropriate internal consistency (Tabachnick & Fidell,

2013).

Results

A summary of the assumption testing for the MOSART data (research question

one), as described in the previous section, is shown in Table 8. Normality for the

experimental group was not tenable. However, no other assumptions were violated.

Table 8

Results of Assumption Testing for Research Question One (MOSART data).

Assumption Result

Measurement of Covariate Covariate

Reliability of the Covariate Cronbach’s α = .98; appropriate (Tabachnick &

Fidell, 2013)

Normality Assumption Not Violated for Control Group

Linearity Assumption Not Violated

Homogeneity of regression slopes Assumption Not Violated

Homogeneity of Variance Assumption Not Violated

Hypothesis Testing

Descriptive Statistics

Descriptive statistics for the MOSART pre-test data are presented in Table 9.

Descriptive statistics for the MOSART post-test data before adjusting for the pre-test data

are presented in Table 10. N = 90 for the MOSART testing, which differs from the

106

previously reported N = 84 for the overall study. Thus, N = 6 did not complete both the

MOSART and CCS posttests or were removed due to outliers or incomplete data as

explained throughout this chapter.

Table 9

Descriptive statistics for the MOSART pre-test data by group.

Group n M SD

Control Group 31 4.81 1.80

Experimental Group 59 6.17 2.31

Table 10

Descriptive statistics for the MOSART posttest data by group.

Group n M SD

Control Group 31 7.39 2.49

Experimental Group 59 5.97 2.58

The posttest data with adjusted means, taking into account the covariate, for the control

group was 7.67 (SD = .46) and the experimental group was 5.82 (SD = .33).

Analysis

After adjusting for the pretest data, the ANCOVA demonstrated that there was a

statistically significant difference between groups at an α = .05 level, F (1, 86) = 7.38, p =

.008, ƞ2 = .08, with an observed power of .77. Since p = .008 is less than α = .05. The

effect size (ƞ2 = .08) is considered a medium to large effect size (Tabachnick & Fidell,

2013), thus indicating a medium to large magnitude of treatment effect (Rovai et al.,

107

2013). The observed power of .77 is near the desired observed power of .8, thus reducing

the likelihood of a Type I error, or rejecting the null hypothesis when it should not be

rejected (Rovai et al., 2013).

Results of Hypothesis One

The first hypothesis stated that there is no statistically significant difference in

middle school students’ misconceptions of science as measured by the MOSART testing

instrument when participating in online collaborative learning as compared to students

who participate in traditional collaborative learning only, while controlling for student

prior knowledge. Results of this study indicated that the first null hypothesis was

rejected. Inspection of the means (control group M = 7.67, SD = .46 and experimental

group M = 5.8, SD = .33) indicated that a statistically significant difference existed

between the posttest scores of the two groups, with the control group’s mean being

greater than the experimental group’s mean; thus indicating that the control group’s mean

scores were higher than the experimental group’s mean scores.

108


Research question two was as follows: Is there a statistically significant

difference in middle school students’ sense of community as measured by the Classroom

Community Scale when participating in online collaborative learning as compared to

students who participate in traditional collaborative learning only? A one-way

multivariate analysis of variance (MANOVA) examines whether multiple dependent

variables are changed when the independent variable is manipulated (Rovai et al., 2013).

Since multiple correlated dependent variables were present in regards to research

question two (the Classroom Community Scale subscales of learning and connectedness),

a MANOVA analysis and follow up analyses were used to analyze the second, third, and

fourth null hypotheses: Ho2: There is no statistically significant difference in middle

school students’ overall sense of community as measured by the Classroom Community

Scale when participating in online collaborative learning as compared to students who

participate in traditional collaborative learning only. H03: There is no statistically

significant difference in middle school students’ connectedness as measured by the

Classroom Community Scale when participating in online collaborative learning as

compared to students who participate in traditional collaborative learning only. Ho4:

There is no statistically significant difference in middle school students’ learning as

measured by the Classroom Community Scale when participating in online collaborative


only. Prior to conducting the analysis, assumption testing was conducted as explained in

the next section.

109

Assumption Testing

An independent t-test was first conducted to ensure that no statistically significant

difference existed among the mean CCS scores of the control and experimental group

(Rovai et al., 2013). Results of the independent t-test indicated that there was no

statistically significant difference in the scores for the control group (M = 48.22, SD =

11.04, n = 27) and the experimental group (M = 44.23, SD = 44.23, n = 57). The

magnitude of the differences in the means (mean difference = 3.99, 95% CI: -.45 to 8.44)

was small to medium (eta square = .038).

The Pearson product-moment correlation coefficient, r, was calculated to

determine if the dependent variables of the Classroom Community Scale (CCS) survey

(the subscales of learning and connectedness) were correlated. Pearson’s r was the most

appropriate test of correlation as the data was Likert-type scale data and Pearson’s r

indicates relationships between the variables (Warner, 2013). A moderate, positive

correlational relationship between the two dependent variables learning and

connectedness, r (83)= .48, p < .05 was found. Given the significant correlations among

the dependent variables, the MANOVA was conducted and deemed appropriate as the

MANOVA considers the interrelationship between variables and determines whether

groups differ on more than one dependent variable (Gall et. al., 2007).

Normality

Normality was tested for through the construction of histograms. Histograms

showed a normal distribution among the CCS data for both the control group and the

experimental group connectedness variable (see Figure 5). However, the histogram for

the experimental group for the learning subscale appeared slightly positively skewed.

110

Figure 5. Histograms for normality testing of research question two.

Normality for the CCS data was also tested for through use of Shapiro-Wilk and

Kolmogorov-Smirnov (Tabachnick & Fidell, 2013). Since the control group contained

less than 50 participants, results of Shapiro-Wilk (Tabachnick & Fidell, 2013) were used

to determine that the control group did not violate assumptions of normality (p = .23 for

learning and p = .221 for connectedness, which were both greater than α = .05). Since

111

the experimental group contained more than 50 participants, results of Kolmogorov-

Smirnoff (Tabachnick & Fidell, 2013) were used to determine that the experimental

group did not violate assumptions of normality for the subscale of Connectedness (p =

.20 which was above α = .05) but did violate assumptions of normality for the subscale of

learning (p = .02 which was below α = .05). However, the test is still considered robust

when the number of participants exceeds 20 (Tabachnick & Fidell, 2013). Further,

inspection of boxplots indicated no extreme outliers.

Multivariate normality for CCS data was examined through Mahalanobis distance

analysis. Analysis showed that the assumption for multivariate normality was not tenable

due to one extreme outlier. One case in the experimental group (case 85) exceeded the

critical value of 13.82 (Tabachnick & Fidell, 2013) and was removed. The data was

found tenable for multivariate normality after the removal of the one case. One case was

removed due to incomplete data. In four cases, the individuals completed the MOSART

pre-test and posttest data but did not complete the CCS pre-test and posttest data. This

resulted in an n = 84 for the CCS analysis.

Linearity

Inspection of a scatterplot of CCS dependent variables indicated that the

assumption of linearity was upheld (see Figure 6). A straight-line relationship existed

between each pair of the dependent variables (Rovai et al., 2013); thus indicating that the

amount or rate of change between scores for the dependent variables of learning and

connectedness were approximately constant for this study. Assumptions of

multicollinearity and singularity were examined through inspection of the scatterplot and

consideration of Pearson’s r (discussed above). Multicollinearity occurs when dependent

112

variables are highly correlated (r = .90 and above) and indicates redundancy of variables,

thus is an important assumption in MANOVA analysis (Rovai et al., 2013). Likewise,

singularity occurs when dependent variables are perfectly correlated (r = 1.00) and

indicates redundancy of variables, thus is an important assumption in MANOVA analysis

(Rovai et al., 2013). Given that r = .48, neither assumption was violated.

Figure 6. Scatterplot of Classroom Community Scale data.

Homogeneity of Variance and Covariance

Homogeneity of variance and covariance is the assumption that two groups have

the same variance (Rovai et al., 2013). Box’s M test was used to determine homogeneity

of covariance for the CCS data. Analysis of the data using Box’s M test indicated that

the assumption of homogeneity of covariance was not violated, F (1, 3) = .18, p = .908,

(Rovai et al., 2013).

The assumption of homogeneity of variance for the CCS data was examined

through the use of Levene’s test. Levene’s test assessed the null hypothesis that the

113

variance of the dependent variables were equal across groups (Rovai et al., 2013).

Results for the learning subscale were F (1, 82) = .002, p = .97. Since p =.97 is greater

than α = .05, the assumption of homogeneity of variance was not violated. Results for the

connectedness subscale were F (1, 82) = .10, p = .75. Since p =.75 is greater than α =

.05, the assumption of homogeneity of variance was not violated.

Results

A summary of the assumption testing for the CCS data (research question two), as

described in the previous section, is shown in Table 11. Normality was slightly

positively skewed for the control group’s learning subscale scores and one extreme

outlier in the experimental group was noted and removed. No additional violations of

assumptions were noted.

Table 11

Results of assumption testing for research question two (CCS data).

Assumption Result

Measurement of Covariate No Covariate

Presence of correlation between the DVs Yes

Normality Assumption Not Violated

Outliers Assumption Not Violated

Multivariate Normality One Extreme Outlier (Removed)

Linearity Assumption Not Violated

Homogeneity of Variance and Assumption Not Violated

Covariance

Multicollinearity Assumption Not Violated

Singularity Assumption Not Violated

Cronbach’s α Cronbach’s α = .80; acceptable

114

Hypothesis Testing

Descriptive Statistics

The N for this analysis was 84. One case was removed due to incomplete data.

One additional case was removed due to extreme outliers. In four cases, the individuals

completed the MOSART pre-test and posttest data but did not complete the CCS pre-test

and posttest data. Descriptive statistics for the CCS pre-test data are presented in Table

12.

Table 12

Descriptive Statistics for the CCS Community Subscale Pre-test Data by Group.

Subscale Group n M SD

Connectedness Control 27 25.70 5.96

Experimental 57 21.67 5.02

Learning Control 27 22.52 7.29


Descriptive statistics for the CCS post-test data are presented in Table 13.

115

Table 13

Descriptive Statistics for the CCS Overall Community Subscale Post-test Data by Group

Subscale Group n M SD

Composite Control 27 48.22 11.04

Community


Connectedness Control 27 25.33 4.93


Learning Control 27 24.04 4.84


Analysis

Wilk’s lambda was used to interpret results of the MANOVA analysis

(Tabachnick & Fidell, 2013). Although sample sizes were different among groups, no

assumptions of homogeneity of variance or covariance were violated as indicated by

Box’s M; thus use of Wilk’s lambda was considered appropriate (Tabachnick & Fidell,

2013). Results of the MANOVA Wilk’s lambda = .91, F (2, 81) = 3.92, p = .02, ƞ2 = .09,

observed power = .69, revealed a significant difference in the composite community

scores between groups. This indicated that the students who engaged in face-to-face

collaborative activities and students who engaged in online collaborative activities did

differ in their sense of community. As reported in the descriptive statistics above, the

means of the control group scores were higher than the means of the experimental group

scores. The effect size (ƞ2 = .09) is considered a medium to large effect size (Tabachnick

& Fidell, 2013), thus indicating a medium to large magnitude of treatment effect (Rovai

116

et al., 2013). Although the observed power of .69 is lower than the desired power of .8,

the observed power is considered reasonable, thus indicating a low probability of a Type

I error (Rovai et al., 2013).

Results of Hypothesis Two

The second hypothesis stated that there is no statistically significant difference in

middle school students’ overall sense of community as measured by the Classroom


students who participate in traditional collaborative learning only. Given the statistical

analysis as explained above, the second null hypothesis was rejected. Since the F statistic

was significant, individual analyses of variance (ANOVAs) for each dependent variable

were performed (Gall et. al., 2007). When results for the dependent variables were

considered separately, a Bonferroni adjusted alpha level of .025 (.05/2) was used to

determine significance to help control for Type I error (Rovai et al, 2013; Tabachnick &

Fidell, 2013).

Hypothesis Three

The third hypothesis stated that there is no statistically significant difference in

middle school students’ connectedness as measured by the Classroom Community Scale


participate in traditional collaborative learning only. An analysis of variance (ANOVA)

was used to analyze the third null hypothesis.

The analysis revealed, F (1, 82) = .08, p = .78, ƞ2 = .001, observed power = .06

that control group M = 25.33, SD = 4.93 and experimental group M = 22.16, SD = 5.41,

did not statistically significant differ in their connectedness scores (see Table 13).

117

The observed power of .06 is lower than the desired power of .8, thus indicating

an increased probability of a Type II error (the researcher can say with only 6 percent

confidence that the correct decision was made) (Rovai et al., 2013).

Results of Hypothesis Three

The third hypothesis stated that there is no statistically significant difference in

middle school students’ connectedness as measured by the Classroom Community Scale


participate in traditional collaborative learning only. This hypothesis was not rejected.

Hypothesis Four

The fourth hypothesis stated that there is no statistically significant difference in

middle school students’ learning as measured by the Classroom Community Scale when


traditional collaborative learning only. An ANOVA was used to analyze the third null

hypothesis.

An ANOVA analysis revealed F (1, 82) = 6.68, p = .01, ƞ2 = .08, observed power

= .72, control group M = 24.04, SD = 4.84 and experimental group M = 23.72, SD = 4.77,

thus indicating that a statistically significant difference in the means was present (see

Table 13). The control group mean scores were higher than the experimental group mean

scores. The effect size (ƞ2 = .08) is considered a medium to large effect size (Tabachnick

& Fidell, 2013), thus indicating a medium to large magnitude of effect (Rovai et al.,

2013). The observed power of .72 is close to the desired power of .8, thus indicating a

reduced probability of a Type I error (Rovai et al., 2013).

118

Results of Hypothesis Four

The fourth hypothesis stated that there is no statistically significant difference in

middle school students’ learning as measured by the Classroom Community Scale when


traditional collaborative learning only. The fourth null hypothesis was rejected.

Summary

Four hypotheses were examined to compare students’ misconceptions of science,

an aspect of science literacy, and students’ sense of community. Mean scores from the

MOSART assessments were analyzed using ANCOVA analysis and the Classroom

Community Scale surveys were analyzed using MANOVA analyses. Results of each

analysis for the corresponding hypothesis are shown in Table 14.

119

Table 14

Results of Statistical Analysis per Hypothesis

Hypothesis Rejected Failed to Reject

Ho1: There is no statistically significant X

difference in middle school students’

misconceptions of science as measured

by the MOSART testing instrument when

participating in online collaborative

learning as compared to students who

participate in traditional collaborative

learning only, while controlling for

student prior knowledge.



overall sense of community as measured

by the Classroom Community Scale when





student community.

H03: There is no statistically significant X


connectedness as measured by the

Classroom Community Scale when





student community.



learning as measured by the Classroom

Community Scale when participating

in online collaborative learning as

compared to students who participate

in traditional collaborative learning

only, while controlling for student

community.

The results showed that there was no statistically significant difference in the

level of science literacy of students who participated in online collaborative learning and

120

students who participated in traditional face-to-face collaborative learning only; thus

hypothesis one was not rejected. Furthermore, the results showed that there was

statistically significant difference in the overall sense of community experienced by

students who participated in online collaborative learning and students who participated

in traditional face-to-face collaborative learning only; thus hypothesis two was rejected.

However, the results showed that there was no statistically significant difference in the

connectedness experienced by students who participated in online collaborative learning

and students who participated in traditional face-to-face learning; thus hypothesis three

was not rejected. The results showed that there was a statistically significant difference

in the learning experienced by students who participated in online collaborative learning

and students who participated in traditional face-to-face learning; thus hypothesis four

was rejected.

Additional Analysis

After assumption testing was deemed acceptable, a mixed between-within

subjects analysis of variance (ANOVA) was conducted to examine the impact of the two

interventions (face-to-face collaboration and online collaboration) on misconceptions as

an aspect of science literacy, as measured by the MOSART assessment, from the pretest

to posttest. An examination of interaction effects is important as a statistically significant

interaction effect may be an indication that the overall pattern of differences across

groups may not be consistent over time. As discussed previously, a statistically

significant difference was found between the control group and experimental group

posttest MOSART scores; with the control group mean scores for the MOSART being

higher than the experimental group mean scores, p = .005. There was also a significant

121

main effect over time, Wilk’s lambda = .86, F (1, 88) = 14.0, p = .000, ƞ2 =.18, observed

power = .138. There was a significant interaction between programs from pretest to

posttest, Wilk’s lambda = .82, F (1, 88) = 19.28, p = .000, ƞ2 =.18, observed power = .99.

Upon inspection of the means (see Table 15), the control group showed a

significant increase in their mean scores from pre-test to posttest; whereas, the

experimental group demonstrated a significant reduction from pretest to posttest.

Table 15

Descriptive Statistics for Additional Analysis

Test Group n M SD

Pre-test Control 31 4.81 1.80


Posttest Control 31 7.39 2.49


The results of this study are important to the current understanding of the effects

of online collaboration on students’ science literacy, given the limited amount of research

within the education literature. Likewise, the results of this study are important to the

current understanding of the effects of online collaboration on students’ sense of

community with a particular emphasis on adolescent students who are participants in the

science classroom. Therefore, the next chapter will discuss the results, the implications,

and the need for future research to broaden educational understanding and knowledge of

best practices as a result of this study.

122

CHAPTER FIVE: DISCUSSION

Introduction

This chapter provides a summary and discussion of the findings of the study,

beginning with the statement of the problem and the purpose of the study. Next, a

summary of the results of each of the research questions is provided and discussed.

Theoretical implications, implications for practice, methodological implications, and

implications for future research are explained. Limitations are discussed and, finally, a

conclusion is made based on the research findings of this study.

Statement of the Problem

Utilizing the conceptual frameworks of constructivism, social development

theory, and community, this quasi-experimental study sought to determine the effects of

online collaborative learning on middle school students’ science literacy as measured by

the Misconceptions-Oriented Standards-Based Resources for Teachers (MOSART)

assessment (Harvard, 2011) and sense of community as measured by Rovai’s (2002a)

Classroom Community Scale.

The independent variable was the type of learning (traditional face-to-face

collaboration or online collaborative learning). Traditional learning was defined as

learning that occurs face-to-face in the classroom. Collaborative learning was defined as

learning that occurs as part of a group where all learners are mutually involved in the

learning process (Bernard, Rubalcava, & St-Pierre, 2000). More specifically, online

collaborative learning was operationally defined as computer-mediated learning that

occurs as part of a group where all learners are mutually involved in the learning process

(Dewiyanti et al., 2007).

123

The dependent variables were middle school student science literacy and student

sense of community. Science literacy was defined as “understanding key scientific

concepts and frameworks, the methods by which science builds explanations based on

evidence, and how to critically assess scientific claims and make decisions based on this

knowledge” (Impey et al., 2011, p. 34), with a specific focus on the identification of

scientific misconceptions (Harvard, 2011). Sense of community was generally defined as

“ a feeling that members have of belonging, a feeling that members matter to one another

and to the group, and a shared faith that members’ needs will be met through their

commitment to be together” (McMillan & Chavis, 1986, p. 9).

Review of Methodology

This study was a quantitative study and used a quasi-experimental pretest/posttest

control group design. This design was most appropriate as the independent variable was

manipulated and a control group was utilized; however, randomization of the sample was

impossible as students were part of pre-existing groups (classes) (Gall et al., 2007).

Since the quasi-experimental design is the next strongest experimental design to true

experimental and true experimental design requires randomization of the sample

population, a quasi-experimental design was utilized (Rovai, Baker, & Ponton, 2013). A

convenience sample of eighth grade physical science students (overall N = 90) at a rural

public middle school in South-Central Virginia were assigned to an experimental and a

control group based on intact pre-existing class assignment. Each group received

equivalent instructional content. However, the experimental group received collaborative

assignments that were completed collaboratively through the use of the Edmodo

educational platform while the control group received assignments that were completed

collaboratively face-to-face. The MOSART (Harvard, 2011) assessment and Sense of

124

Community Scale (Rovai, 2002a) survey were administered to all students prior to the

treatment and post-treatment. Results were statistically analyzed and reported.

Summary of Results


Research question one was as follows: Is there a statistically significant

difference in middle school students’ misconceptions, aspect of science literacy, as

measured by the MOSART testing instrument when participating in online collaborative


only? Prior to the primary analysis, an independent t-test was used to determining if

there was a statistically significant difference in pre-test scores across groups and the

need to control for the covariate was present. An analysis of covariance (ANCOVA)

analysis was used to examine whether a statistically significant difference existed

between the control group and experimental group MOSART scores. Results indicated

that there was a statistically significant difference in middle school students’



traditional collaborative learning only. Examination of the mean MOSART scores

between groups indicated that the control group’s MOSART scores were higher than the

experimental group’s MOSART scores; thus, students participating in face-to-face

collaboration experienced higher levels of science literacy (or, alternatively, reduced

misconceptions) than students participating in online collaboration.

125


Research question two was as follows: Is there a statistically significant

difference in middle school students’ sense of community as measured by the Classroom


students who participate in traditional collaborative learning only? An independent t-test

was used to determining if there was a statistically significant difference in pre-test scores

across groups and the need to control for the covariate was not present. A multivariate

analysis of variance (MANOVA) was used to examine whether a statistically significant

difference existed between the control group and experimental group overall CCS scores.

Results indicated that a statistically significant difference did exist in middle school

students’ sense of community as measured by the Classroom Community Scale when

participating in online collaborative learning as compared to students who participated in

traditional collaborative learning only. Examination of the mean CCS scores between

groups indicated that the control group’s CCS scores were higher than the experimental

group’s CCS scores; thus, students participating in face-to-face collaboration experienced

a higher sense of community compared to students participating in online collaboration.

A follow up analysis of variance (ANOVA) was used to examine whether a

statistically significant difference between the control group and experimental group

connectedness existed. Results indicated that no statistically significant difference

existed between middle school students’ connectedness as measured by the Classroom


students who participate in traditional collaborative learning only.

126

An analysis of variance (ANOVA) was also used to examine whether a

statistically significant difference between the control group and experimental group

learning existed. Results indicated that a statistically significant difference existed

between middle school students’ learning as measured by the Classroom Community

Scale when participating in online collaborative learning as compared to students who

participate in traditional collaborative learning only. Examination of the mean learning

scores between groups indicated that the control group’s learning scores were higher than

the experimental group’s learning scores; thus, students participating in face-to-face

collaboration experienced an increased sense of learning compared to students

participating in online collaboration.

Additional Analysis

A mixed between within subjects analysis of variance (ANOVA) was conducted

to examine the impact of the two interventions (face-to-face collaboration and online

collaboration) on misconceptions as an aspect of science literacy, as measured by the

MOSART assessment, from the pretest to posttest. Both main effects and the interaction

effect were found significant. The control group showed a significant increase in their

mean scores on the MOSART from pre test to post test; whereas, the experimental group

demonstrated a significant reduction in their mean scores on the MOSART from pretest

to posttest.

Discussion of Results


The difference in the mean scores on the MOSART, with the face-to-face group

scoring higher than the online group, can be understood in light of the research on

Computer Mediated Communication (CMC) and the challenged documented for

127

asynchronous communication provide an explanation for the significant difference. Four

qualities have been identified that are fundamental for CMC technologies: temporality,

identity, modality, and spatiality (Zhao et al., 2004). In synchronous learning, individuals

are able to communicate directly and immediately with other individuals, thus engaging

in real-time discussion and receiving immediate feedback. In asynchronous learning, the

communication of individuals is limited to a one-way channel, thus discussion and

feedback are delayed. While some research has demonstrated the benefits of these

features of online asynchronous learning, the challenges are well documented also.

The delayed feedback and interaction of asynchronous communication requires

the learner to “back up to answer a question” once they may have moved onto another

discussion (Zhao et al., 2004, p. 27). Miscommunication can occur and discussion may

become confusing if multiple replies on multiple topics are posted at different times

(Conrad & Donaldson, 2004). Ahern et al. (2006) found that communication that

requires time, such as that afforded in asynchronous communication, increases the

difficulty in engaging in dialogue and peer-to-peer interactions and the quality of

interactions that require higher order thinking skills may be reduced (Kanuka, Rourke, &

Laflamme, 2007). Although asynchronous communication methods create a sense of

anonymity that may increase students’ attention on the responses of fellow students, it at

the same time decreases the quality of students’ work (Zhao et al., 2004). In the current

study, students participating in online discussion may have experienced these

phenomenon and, thus, had a lower MOSART score than the students participating in

face-to-face discussion.

128

Further, the finding may be explained by the lack of nonverbal cues within an

asynchronous text based environment. In research comparing face-to-face and online

learning environments, Meyer (2007) found that students not only preferred the face-to-

face environment but also were able to “capture the feel, tone, and emotion” (p. 66) of

communication exchanges, thus leading to an increase in retention. Asynchronous

discussion, therefore, may limit verbal and social cues that are necessary for effective

communication, and ultimately, learning. Garrison et al. (2001) suggested that the

asynchronous, text based medium may “not support this [resolution response] kind of

activity” (p. 13), and Thomas (2002) suggested that “the attainment of a discourse that

is…academic in nature is difficult within the online environment of the traditional

threaded discussion” (p. 359). The collaborative science activities may have been better

suited for the face-to-face environment rather than the online environment.

Additional analysis for the study also indicated that the overall mean MOSART

scores of the experimental group significantly decreased from pre-test to posttest,

indicating that online collaborative learning led to an increase in misconceptions (or a

decrease in science literacy). The CoI framework provides some insight into this finding.

Lack of teaching presence in design and instruction are attributable to low levels of

learning or lack of critical thinking (Garrison, et al, 2001). The lack of immediate teacher

feedback, due to the asynchronous nature of the online learning environment, may have

provided an increased opportunity for reinforcement of peer misconceptions rather than

immediate redirection and reduction of student misconceptions. The synchronous nature

of the face-to-face collaborative learning environment provided opportunities for

immediate teacher feedback, thus reducing peer generated misconceptions. In the

129

asynchronous online collaborative learning environment, students participated in online

activities with one another, and the teacher discussed them and corrected misconceptions

in the following days’ instruction. Teacher’s redirection and corrections of

misconceptions were delayed and occurred after significant number of peer interactions

that sometimes supported a misconception. The findings of this study support research

that indicates the importance of teacher immediacy and presence in online learning

environments. It is possible that decreased teacher immediacy in the experimental group

may have impacted the results of this study. Thus, increased teacher immediacy in future

studies may provide different results. Teacher presence has been found to be an

important aspect of the quality of discussions and has been found to be lacking in

asynchronous environments, thus leading to a breakdown in communication (Kucuk,

2009). Additionally, the findings of this study support the proposal that asynchronous

environments may make it difficult to collaborate as a group and negotiate responses

(Garrison & Anderson, 2003).


Sense of community is multi-dimensional. Connectedness is defined as “the

feeling of belonging and acceptance and the creation of bonding relationship” (Rovai,

2002, p. 322). This study indicated that no difference existed between groups

participating in face-to-face learning and online learning in terms of connectedness, thus

indicating that the learning environment for each group fostered feelings of belonging

and acceptance.

However, students who participated in face-to-face collaboration experienced

higher gains in the learning aspect of community than those who participated in online

130

collaboration. Learning is defined as “the feeling that knowledge and meaning are

actively constructed within the community, that the community enhances the acquisition

of knowledge and understanding, and that the learning needs of its members are being

satisfied” (Rovai, 2002, p. 322). This aspect of community is correlated with critical

thinking and learning outcomes; research on sense of community has supported that an

increased sense of community may facilitate increased learning outcomes (Rovai, 2002;

Rovai, Wighting, & Liu, 2005). This was supported by this study as students

participating on face-to-face collaboration experienced increased learning community as

well as increased learning as measured by level of misconceptions by the MOSART

assessment.

Although some research has demonstrated that sense of community is equivalent

for online and face-to-face learners at the higher education level (Rovai, 2002; Rovai,

Wighting, & Liu, 2005), adolescent sense of community experiences are different given

the changing relationships and life experiences that occur as students transition to

adulthood. Opportunities to influence and interpret social roles, experience power, and

interact are key to the experience of sense of community (Chiessi, Cicognani, & Sonn,

2010), and adolescents given their development may have difference experiences in

creating online community as compared to adults.

Results of this study indicated that students’ overall sense of community was

higher when engaging in face-to-face collaborative learning as compared to online

collaborative learning. These results are explained by the research that suggests that

some students prefer face-to-face communication as computer-mediated communication

decreases the individual’s ability to determine how others feel and decrease likelihood of

131

consensus (Palloff & Pratt, 1999) as indicated by the greater gains in sense of community

exhibited by students who participated in face-to-face collaboration as compared to those

who participated in online collaboration. Further, nonverbal cues, human reassurance,

and robustness of dialogue may be present in the face-to-face environment but lacking in

the online environment (Vaughan & Garrison, 2005). Face-to-face communication

facilitates the process of negotiation and resolution of conflict, both of which are

necessary for group cohesiveness and the forming of connectivity. Through synchronous

face-to-face discussion, alternative ideas may be expressed, questioned, and supported,

reducing unresolved conflict that may not be appropriately addressed in asynchronous

online environments, thus leading to decreased community (Palloff & Pratt, 1999).

Additionally, research on community has found that “a major challenge facing

educators using CMC is the creation of the critical community of inquiry…within a

virtual text based environment” (Garrison et al., 2001, p. 1). This study, therefore,

supports that asynchronous learning environments pose challenges in creating community

in the middle school environment.

Implications

Theoretical Implications

The results of this study support social development theory, which

purports that individuals learn through social experiences (Vygotsky, 1986). Given that

the control group mean MOSART scores increased from pre-test to posttest, this study

indicates that face-to-face collaborative learning in a social learning environment

decreases misconceptions of science. This upholds the tenets of social development

theory that an increase in learning occurs through social learning activities that occur in

132

the face-to-face format. Considering that overall group means of CCS scores increased

for both the control and experimental group, this study supports that collaborative

learning results in an increase in sense of community, thus upholding the theory of

communities of practice (Wenger, 1998); however, the control group had a higher sense

of community and mean MOSART scores than the treatment group.

In regards to community of inquiry theory, the effective educational experience

occurs when cognitive presence, social presence, and teaching presence overlap

(Garrison, Anderson, & Archer, 2010); thus, suggesting that face-to-face collaborative

learning environment involved appropriate cognitive presence, social presence, and

teaching presence, whereas the online collaborative learning environment did not. This

may be further explained by media richness theory, which purports that face-to-face

communication has advantages over other forms of communication in which immediacy

of feedback is delayed (Daft, Lengel, Trevino, & Kiebe, 1987). Loss of nonverbal cues

may lead to a breakdown in communication, thus resulting in less effective learning.

With the synchronous nature of the face-to-face collaborative environment, students were

able to receive immediate feedback from peers and the teacher. Nonverbal

communication was readily observed and led to an increase in understanding. However,

with the asynchronous nature of the online collaborative environment, feedback from

peers and the teacher was not immediate, but rather delayed. This coupled with the lack

of nonverbal cues may have resulted in miscommunication; thus lower levels of learning

and confusion. The treatment groups’ decrease in MOSART scores, an increase in

student misconceptions, as compared to the control group’s increase in MOSART score,

133

a decrease in misconceptions, can be explained by and confirms Media Richness theory .

A summary of these findings are provided in Table 16.

Table 16

Description of organization of theoretical framework, research questions, design, and

data with outcomes

Research Theoretical Data Sources Outcomes Contribution

Question Framework

RQ 1 Social MOSART Increased MOSART Supports Social

Development scores of the control Development Theory

Theory group

Media MOSART Higher MOSART Supports Media

Richness scores for the control Theory

Theory group over the

experimental group

RQ 2 Communities Classroom Higher Sense Supports

of Practice Community of Community Communities of

Theory Scale for the control Practice Theory

group over the

experimental group

Media Classroom Higher Sense Supports Media

Richness Community of Community Richness Theory

Theory Scale in the Face-to-

Face Environment

Limitations

Several limitations existed in this study. Non-randomization was a limitation of

this study (Rovai et al., 2013). The lack of randomization of the study provides a slightly

weaker design than desirable and becomes an internal threat to validity (Rovai et al.,

134

2013). Since randomization of the sample was not possible in this study due to intact

groups (classes), a quasi-experimental design was used. A pretest was, therefore,

administered to assist in controlling for lack of randomization (Campbell & Stanley,

1963). Use of a pretest addressed the internal threats of selection, participant history,

maturation, and regression (Rovai et al., 2013); however, it introduced the testing threat

to validity.

Generalizability may have been a limitation of this study (Rovai et al., 2013).

The results of the study may not be generalizable to other populations or grade levels, or

may not be generalizable to other subject areas within the science field. It was assumed

that the sample population is representative of all middle school science students in

Virginia. However, this may not be the case and leads to external threats of validity.

Further studies to determine generalizability would need to be conducted including future

longitudinal studies.

Student history may have been a limitation of this study (Rovai et al., 2013). The

prior knowledge of students may not have been the same. This presented a threat to

internal validity. Therefore, prior knowledge was statistically controlled for through the

use of a pretest-posttest design to reduce threat to internal validity (Gall et al., 2007).

Non-equivalence of groups however was the primary limitation of this study

(Rovai et al., 2013). Although measures were taken to control for this threat to validity

thorough the use of a pretest and homogenous groups, the threat still existed (Campbell &

Stanley, 1963).

Participant non-accordance with prescribed research guidelines may have been a

limitation of this study. Students were assumed to have correctly follow guidelines

135

presented by the respective classroom teacher. Specifically, those in the experimental

group were assumed to have completed assignments collaboratively using a computer

and those in the control group were assumed to have completed assignments

collaboratively in a traditional or face-to-face manner. Experimental treatment diffusion,

when communication occurs between groups, may have been a limitation of this study

(Rovai et al., 2013), thus students were instructed to have no communication between the

groups. An access code to Edmodo was provided for students in the experimental group,

thus preventing access by the control group.

Implementation may have been a limitation of this study (Rovai et al., 2013). It is

possible that students in the experimental and control groups may have been treated

differently by the two classroom teachers and may have been provided with different

experiences despite efforts to reduce this likelihood. Since both groups were subject to

the same curriculum requirements and pacing guides, it was assumed that all instructional

content provided to the experimental group and the control group was equivalent,

therefore providing treatment fidelity. This included both in-class face-to-face

instructional content as well as the content of the collaborative activities. Two classroom

teachers were used for this study. However, to ensure treatment fidelity, the teachers

were instructed to provide equivalent instruction to both groups as provided by the

county curriculum and pacing guide. Strict instructions to the classroom teachers

regarding the need to retain homogenous instruction for multiple classes was provided by

the researcher. In addition, training was provided to the classroom teacher assigned to

the experimental group on the use of Edmodo prior to beginning the study.

Finally, the treatment fidelity of the MOSART assessment and Classroom

136

Community Scale may have provided an additional threat to internal validity. It is

possible that the MOSART assessment and Classroom Community Scale may have been

administered by the classroom teachers differently among the groups. Measures to

ensure treatment fidelity were taken, including requiring the classroom teachers to

complete online tutorials on the administration and use of the MOSART assessment,

providing a script to ensure that administration of the MOSART assessment was the same

for each group, and providing a script to ensure that administration of the Classroom

Community Scale survey was the same for each group.

Implications for Practice and Methodological Implications

The mode of instruction has also been shown to be an indicator of the quality of

communication and, thus, the quality of online learning (Conrad & Donaldson, 2004).

Some tasks are better suited for some modes of instruction than others; technology is no

exception (Zhao et al., 2004). The teacher must choose the mode of technology to suit

the learning task (Conrad & Donaldson, 2004). Results of this study demonstrated that

collaborative learning activities may have been best-suited for face-to-face learning in

future practice and study.

Thus, current practices of encouraging technology implementation in the middle

school classroom may not produce positive benefits for students as students participating

on face-to-face collaboration experienced increases in MOSART scores as well as sense

of community whereas those participating in online collaboration experienced decreases

in MOSART scores and a smaller increase in sense of community. As indicated by the

decrease in MOSART scores for students participating in online collaboration,

technology implementation may actually prove to detrimental to reducing misconceptions

137

in the science classroom. Thus, prior to integrating technology as a tool in the classroom,

further study should be conducted to determine advantages and disadvantages in the

science classroom among the adolescent population.

In this, consideration of the teacher’s role in structuring learning tasks to most

effectively meet the goals of social presence, cognitive presence, and teaching presence

must also occur prior to technology implementation in any online learning environment.

As evidenced by this study, students in the face-to-face environment who experienced

greater teacher presence and immediacy of feedback exhibited greater increases in

MOSART scores as well as sense of community as compared to those who participated in

the online environment. Thus, recommendations for future practice to increase teacher

presence include providing examples and opportunities for developing the elements of

setting climate, supporting discourse, and selecting content to foster learning and

providing feedback and ongoing support for instructors (Swan 2004). Furthermore,

increasing teacher verbal immediacy, especially in the asynchronous environment, may

lead to increased teacher presence and increased sense of community (Ni & Aust, 2008).

Implications for Future Research

Future research should focus on replication of the results of this study as well as

study of the effects of online collaboration as compared to face-to-face collaboration in

other science areas. A variety of MOSART assessments are available for teacher use in

the areas of life sciences, earth science, physics, and chemistry; thus, in order to

determine the generalizability of the results found in this study, further study is needed

that makes use of different MOSART assessments. Further study may also explore the

generalizability of this study to other populations as the students who participated in this

138

study were primary Caucasian and were from a rural public school. Additional study

may examine if similar results are found with other ethnicities, with suburban or urban

communities, and public and private schools.

Further study may consider the role of the teacher in providing feedback; that is,

the immediacy of the feedback, and how immediacy may influence student sense of

community and reduction of misconceptions with a media richness theoretical

framework. This may provide a greater understanding as to why the MOSART scores of

students participating in online collaboration decreased while those of the students

participating in face-to-face communication increased and how teacher immediacy is

related to sense of community (Ni & Aust, 2008) and student participation (Kucuk,

2009). Specifically, a study that examines the relationship between the frequency of

teacher interaction and student knowledge is recommended.

Further study in the area of misconceptions may also provide information related

to how misconceptions may be most effectively reduced in the science classroom.

Research has shown that student misconceptions may increase with computer mediated

learning (Tutty & Klein, 2008) as supported in this study; thus, further investigation to

identify methods of decreasing student misconceptions in the science classroom would be

beneficial.

Research has supported that teachers must encourage group cohesion so that all

members of the group feel obligated to participate in order to improve online education

(Ahern et al., 2006). The results of this study support this conclusion and also provide

further implications to study instructional design and online collaborative design,

139

ensuring that teachers monitor students appropriately to foster complete participation as

well as encourage group cohesion.

In regards to research design, future methodology may include a non-random

sample as this study employed a convenience sample of students in order to strengthen

the design of the study (Gall, Gall, & Borg, 2007). In addition, a true experimental

design could be employed rather than a quasi-experimental design, thus increasing the

strength of the experimental design and validity of the results (Gall, Gall, & Borg, 2007).

Conclusion

The purpose of this study was to determine the effect of online collaboration on

middle school students’ sense of community and science literacy. Results indicated that

there was a statistically significant difference in sense of community and science literacy

of students participating on online collaborative learning as compared to face-to-face

collaborative learning, with students participating in face-to-face collaborative learning

experiencing higher sense of community and levels of science literacy. Furthermore,

results of this study indicated that there was a statistically significant difference in sense

of learning of students participating in online collaborative learning as compared to face-

to-face collaborative learning, with students participating in face-to-face collaborative

learning experiencing higher sense of learning. Results of this study indicated that there

was no statistically significant difference in sense of connectedness. Based on these

results, traditional face-to-face collaboration was found to produce an increase in positive

student outcomes as compared to online collaboration, suggesting the need for further

examination of current pedagogy utilizing technology in the middle school classroom and

140

increased attention to feedback immediacy to foster student sense of community and

reduction of misconceptions of science.

141

References

Abfalter, D., Zaglia, M. E., & Mueller, J. (2012). Sense of virtual community: A follow

up on its measurement. Computers in Human Behavior, 28(2), 400-404.

Admiraal, W., & Lockhorst, D. (2012). The Sense of Community in School Scale

(SCSS). Journal of Workplace Learning, 24(4), 245-255. doi:

10.1108/13665621211223360

Ahern, T. C., Thomas, J. A., Tallent-Runnels, M. K., Lan, W. Y., Cooper, S., Lu, X., &

Cyrus, J. (2006). The effect of social grounding on collaboration in a computer-

mediated small group discussion. Internet and Higher Education 9, 37-46.

American Association for the Advancement of Science. (2013). Benchmarks for science

literacy. Retrieved from http://www.project2061.org/publications/bsl/online/

index.php

American Association for the Advancement of Science. (1991). Benchmarks for science

literacy: Project 2061. New York: Oxford University Press.

American Association for the Advancement of Science. (1990). Science for all

Americans. New York: Oxford University Press.

Anderson, C. W. (2007a). In S. K. Abell & N. G. Lederman (Eds.) Handbook of research

on science education (pp. 3-30). New York: Routledge.

Anderson, R. D. (2007b). Inquiry as an organizing theme for science curricula. In S. K.

Abell & N. G. Lederman (Eds.) Handbook of research on science education (pp.

807-830). New York: Routledge.

http://dx.doi.org.ezproxy.liberty.edu:2048/10.1108/13665621211223360

142

Akinbobola, A. O., & Afolabi, F. (2009). Constructivist practices through guided

discovery approach: The effect on students’ cognitive achievements in Nigerian

senior secondary school physics. Bulgarian Journal of Science and Education

Policy, 3(2), 233-252.

Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J.

C., & Swan, K. P. (2008). Developing a community of inquiry instrument:

Testing a measure of the Community Inquiry framework using a multi-

institutional sample. Internet and Higher Education, 11(3), 133-136.

Baker, P. H., & Murray, M. M. (2011). Building community partnerships: Learning to

serve while learning to teach. The School Community Journal, 21(1), 113-127.

Baker-Doyle, K. J., & Yoon, S. A. (2011). In search of practitioner-based social capital:

A social network analysis tool for understanding and facilitating teacher

collaboration in a US-based STEM professional development program.

Professional Development in Education, 37(1), 75-93.

Bell, T., Urhahne, D., Schanze, S., & Ploetzner, R. (2010). Collaborative inquiry

learning: Models, tools, and challenges. International Journal of Science

Education, 32(3), 349-377.

Bergtrom, G. (2011). Content vs. learning: An old dichotomy in science courses. Journal

of Asynchronous Learning Networks, 15(1), 34-45.

Bernard, R. M., Rubalcava, B. R., & St-Pierre, D. (2000). Collaborative online distance

learning: Issues for future practice and research. Distance Education, 21(2), 260-

277.

143

Black, A. (2010). Gen Y: Who they are and how they learn. Educational Horizons, 88(2),

92-101.

Bluic, A., Ellis, R., Goodyear, P., & Piggott, L. (2010). Learning through face-to-face

and online discussions: Associations between students’ conceptions, approaches,

and academic performance in political science. British Journal of Educational

Technology, 41(3), 512-524.

Briggs, S. R., & Cheek, J. M. (1986). The role of factor analysis in the evaluation of

personality scales. Journal of Personality, 54, 106-148.

Buch, K., & Spaulding, S. (2011). The impact of a psychology learning community on

academic success, retention, and student learning outcomes. Teaching of

Psychology, 38(2), 71-77.

Buraphadeja, V. & Kumnuanta, J. (2011). Enhancing the sense of community and

learning experience using self-paced instruction and peer tutoring in a computer-

laboratory course. Australasian Journal of Educational Technology, 27(8), 1338-

1355.

Burgoon, J. N., Heddle, M. L., & Duran, E. (2011). Re-examining the similarities

between teacher and student conceptions about physical science. Journal of

Science Teacher Education, 22(2), 101-114.

Bye, L, Smith, S., & Rallis, H. M. (2009). Reflection using an online discussion forum:

Impact on student learning and satisfaction. Social Work Education, 28(8), 841-

855.

Cameron, B. A., Morgan, K., Williams, K. C., & Kostelecky, K. L. (2009). Group

projects: Student perceptions of the relationship between social tasks and a sense

144

of community in online group work. American Journal of Distance Education,

23(1), 20-33.

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs

for research. Boston, MA: Houghton Mifflin Company.

Cavas, P. (2011). Factors affecting the motivation of Turkish primary students for science

learning. Science Education International, 22(1), 31-42.

Center for School Reform at TERC. (2012). MOSART profile. Retrieved from

http://mosart.mspnet.org/index.cfm/profile

Chelliah, J., & Clarke, E. (2011). Collaborative teaching and learning: Overcoming the

digital divide? On the Horizon, 19(4), 276-285.

Cheung, C. M. K., Chui, P., & Lee, M. K. O. (2011). Online social networks: Why do

students use Facebook? Computers in Human Behavior, 27(1), 1337-1343.

Chiessi, M., Cicognani, E., & Sonn, C. (2010). Assessing sense of community on

adolescents: Validating the brief scale of Sense of Community in Adolescents

(SOC-A). Journal of Community Psychology, 38(3), 276-292.

Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale

development. Psychological Assessment, 7, 309-319.

Clark, R. C. & Mayer, R. E. (2011). E-Learning and the science of instruction. San

Francisco: Pfeiffer.

Clary, R. M., & Wandersee, J. H. (2012). Mandatory climate change discussions in

online classrooms: Promoting students’ climate literacy and understanding the

nature of science. Journal of College Science Teaching, 41(5), 70-79.

145

Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational

Technology Research and Development, 45(4), 21-35.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ:

Lawrence Erlbaum Associates.

Common Core State Standards Initiative. (2012). The standards: English language arts

standards. Retrieved from http://www.corestandards.org/the-standards/english-

language-arts-standards

Conrad, R. & Donaldson, J. A. (2004). Engaging the online learner. San Francisco:

Jossey-Bass.

Daft, R L., Lengel, R. H., & Trevino, L. K. (1987). Message equivocality, media

selection, and manager performance: Implications for information systems. MIS

Quarterly, 11(3), 356-366.

Dawson, S. (2008). A study of the relationship between student social networks and sense

of community. Educational Technology & Society, 11(3), 224-238.

Dewey, J. (1922). Democracy and education. New York: The Macmillan Company.

Dewey, J. (1997). Experience and education. New York: Simon & Schuster.

Dewiyanti, S., Brand-Gruwel, S., Jochems, W., & Broers, N. J. (2007). Students’

experiences with collaborative learning in asynchronous computer-supported

collaborative learning environments. Computers in Human Behavior, 23(1), 496-

514.

Ding, N., & Harskamp, E. G. (2011). Collaboration and peer tutoring in chemistry

laboratory education. International Journal of Science Education, 33(6), 839-863.

146

Donne, V. (2012). Wiki: Using the web connections to connect students. TechTrends,

56(2), 31-36.

Drouin, M. A. (2008). The relationship between students’ perceived sense of community

and satisfaction, achievement, and retention in an online course. The Quarterly

Review of Distance Education, 9(3), 267-284.

Edmodo. (2012). Edmodo: About. Retrieved from http://about.edmodo.com/

Elsaadani, M. (2012). Exploration of teaching staff and students’ preferences of

information and communication technologies in private and academic lives.

International Journal of Computer Science Issues, 9(2), 396-402.

Evans, R. R., & Forbes, L. (2012). Mentoring the ‘Net Generation’: Faculty perspectives

in health education. College Student Journal, 46(2), 397-404.

Evans, S. D. (2007). Youth sense of community: Voice and power in community

contexts. Journal of Community Psychology, 35(6), 693-709.

Fang, Z., & Wei, Y. (2010). Improving middle school students’ science literacy through

reading infusion. The Journal of Educational Research, 103(4), 262-273.

Fata-Hartley, C. (2011). Resisting rote: The importance of active learning for all course

learning objectives. Journal of College Science Teaching, 40(3), 36-39.

Ferreira, M. M. & Trudel, A. R. (2012). The impact of problem-based learning (PBL) on

student attitudes toward science, problem-solving skills, and sense of community

in the classroom. Journal of Classroom Interaction, 47(1), 23-30.

Findlay, H. J. (2012). Cognitive neuroscience learning theories coupled with

technologies: A conduit for deep and lasting learning. Journal of Applied

Learning Technology, 2(1), 27-31.

147

Fraser, B. J. (2007). Classroom learning environments. In S. K. Abell & N. G. Lederman

(Eds.) Handbook of research on science education (pp. 103-124). New York:

Routledge.

Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction.

Boston: Pearson.

Gardner, J. (1996). Building community. Community Education Journal, 23(3), 6-9.

Garrison, D. R., & Anderson, T. (2003). E-Learning in the 21st century: A framework for

research and practice. London: Routledge/Falmer.

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive

presence, and computer conferencing in distance education. American Journal of

Distance Education, 15(1), 7-23.

Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community

of inquiry framework: A retrospective. Internet and Higher Education, 13(1), 5-9.

Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry

framework: Review, issues, and future directions. Internet and Higher Education,

10(3), 157-172.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative

potential in higher education. Internet and Higher Education, 7(2), 95–105.

Glynn, T. J. (1981). Psychological sense of community: Measurement and application.

Human Relations, 34(9), 780-818.

Gunawardena, C. N., & McIsaac, M. (2004). Distance education. In D. H. Jonassen (Ed.)

Handbook of research for educational communications and technology (2nd

ed.,

pp. 355-394). Mahwah, NJ: Taylor & Francis.

148

Guo, C. (2007). Issues is science learning: An international perspective. In S. K. Abell &

N. G. Lederman (Eds.) Handbook of research on science education (pp. 227-256).

New York: Routledge.

Harvard-Smithsonian Center for Astrophysics. (2012). Science education department.

Retrieved from http://www.cfa.harvard.edu/sed/projects/mosart.html

Harvard College. (2011). MOSART self-service site: Misconceptions-Oriented

Standards-Based Assessment Resources for Teachers. President and Fellows of

Harvard College. Retrieved from http://www.cfa.harvard.edu/smgphp/mosart/

testinventory_2.html

Hillery, G. A. (1955). Definitions of community: Areas of agreement. Rural Sociology,

20(2), 111-123.

Hsu, P., & Roth, W. (2010). From a sense of stereotypically foreign to belonging in a

science community: Ways of experiential descriptions about high school students’

science internship. Research in Science Education, 40(3), 291-311.

Impey, C., Buxner, S., Antonellis, J., Johnson, E., & King, C. (2011). A twenty-year

survey of science literacy among college undergraduates. Journal of College

Science Teaching, 40(4), 31-37.

Institute for Defense Analyses, Science & Technology Policy Institute. (2006). The

National Defense Education Act of 1958: Selected outcomes (IDA Document D-

3306). Retrieved from https://www.ida.org/upload/stpi/pdfs/ida-d-3306.pdf

Institute of Education Sciences, U.S. Department of Education. (2012). The Nation’s

Report Card: Science 2011. Retrieved from http://nces.ed.gov/nationsreportcard/

pubs/main2011/2012465.asp

149

International Technology and Engineering Educators Association. (2011). Technology for

All Americans Project. Retrieved from http://www.iteea.org/TAA/TAA.html.

Jeong, H., & Chi, M. T. H. (2007). Knowledge convergence and collaborative learning.

Instructional Science, 35(4), 287-315.

Jewell, E. J. (Ed.). (2002). Oxford American desk dictionary and thesaurus. New York:

Oxford University Press.

Johnson, D. W., & Johnson, R. T. (2009). An educational psychology success story:

Social interdependence theory and cooperative learning. Educational Researcher,

38(5), 365-379.

Johnson, D. W., & Johnson, R. T. (1996). Cooperation and the use of technology. In D.

Jonassen (Ed.), Handbook of research for educational communications and

technology (pp.785-812). London: MacMillan.

Kanuka,. H., Rourke, L., & Laflamme, E. (2007). The influence of instructional methods

on the quality of online discussion. British Journal of Educational Technology,

38(2), 260-271.

Kelly, C. (2011). A critical pedagogy of cafeterias and communities: The power of

multiple voices in diverse settings. Middle Grades Research Journal, 6(2), 97-

111.

Kelly, G. J. (2007). Discourse in science classrooms. In S. K. Abell & N. G. Lederman

(Eds.) Handbook of research on science education (pp. 443-469). New York:

Routledge.

150

Keser, H., Uzunboylu, H., & Ozdamli, F. (2011). The trends in technology supported

collaborative learning studies in 21st century. World Journal on Educational

Technology, 3(2), 103-119.

Koh, M. H., & Hill, J. R. (2009). Student perceptions of group work in an online course:

Benefits and challenges. Journal of Distance Education, 23(2), 69-92.

Koh, E., & Lim, J. (2012). Using online collaboration applications for group assignments:

The interplay between design and human characteristics. Computers & Education,

59(2), 481-496.

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational

Technology, Research, and Development, 42(2), 7-19.

Kucuk, M. (2009). Teacher immediacy behaviors and participation in computer mediated

communication. Turkish Online Journal of Distance Education, 10(2), 225-235.

Lau, K. (2009). A critical examination of PISA’s assessment on scientific literacy.

International Journal of Science and Mathematics Education, 7(6), 1061-1088.

Laugksch, R. C. (2000). Scientific literacy: A conceptual overview. Science Education,

84(1), 71-94.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.

New York: Cambridge University Press.

Levene, H. (1960). Robust tests for equality of variances. In I. Olkin (Eds.) Contributions

to probability and statistics (pp. 278-291). Stanford, CA: Stanford University

Press.

Lightfoot, J. M. (2009). Student communication preferences in a technology-enhanced

learning environment. International Journal of Instructional Media, 36(1), 9-20.

151

Lim, J., Yang, Y. P., & Zhong, Y. (2009). Group support systems as collaborative

technologies: A meta-analysis. In N. Karacapilidis (Eds.) Solutions and

innovations in web-based technologies for augmented learning: Improved

platforms, tools, and applications (pp.79-108). New York: Information Science

Reference.

Liu, X. (2009). Beyond science literacy: Science and the public. International Journal of

Environmental & Science Education, 4(3), 301-311.

Lou, Y., Abrami, P. C., & d’Apollonia, S. (2001). Small group and individual learning

with technology: A meta-analysis. Review of Educational Research, 71(3), 449-

521.

Luft, J., Bell, R. L., & Gess-Newsome, J. (2008). Science as inquiry in the secondary

setting. Arlington, VA: NSTA Press.

Lunetta, V. N., Hofstein, A., & Clough, M. P. (2007). Learning and teaching in the

school of science laboratory: An analysis of research, theory, and practice. In S.

K. Abell & N. G. Lederman (Eds.) Handbook of research on science education

(pp. 393-441). New York: Routledge.

Mahle, M. (2011). Effects of interactivity on student achievement and motivation in

distance education. The Quarterly Review of Distance Education, 12(3), 207-215.

Mäkitalo-Siegl, K., Kohnle, C., & Fischer, F. (2011). Computer-supported collaborative

inquiry learning and classroom scripts: Effects on help-seeking processes and

learning outcomes. Learning and Instruction, 21(2), 257-266. doi:

10.1016/j.learninstruc.2010.07.001

152

McMillan, D. W., & Chavis, D. M. (1986). Sense of community: A definition and theory.

Journal of Community Psychology, 14(1), 6-23.

Meyer, K. A. (2007). Student perceptions of face-to-face and online discussions: The

advantage goes to.... Journal of Asynchronous Learning Networks, 11(4), 53-69.

Miller, J. D. (2007). The public understanding of science in Europe and the United

States. Paper presented at the 2007 Annual Meeting of the American Association

for the Advancement of Science, San Francisco, CA. Retrieved from

http://ucll.msu.edu/files_ucll.msu.edu/docs/miller-science-europe.doc

Miller, R. L., & Benz, J. J. (2008). Techniques for encouraging peer collaboration:

Online threaded discussion or fishbowl interaction. Journal of Instructional

Psychology, 35(1), 87-93.

Minocha, S. (2009a). A case study-based investigation of students’ experiences with

social software tools. New Review of Hypermedia and Multimedia, 15(3), 245-

265.

Minocha, S. (2009b). Role of social software tools in education: A literature review.

Education & Training, 51(5/6), 353-369.

Moolenaar, N. M., Sleegers, P. J. C., & Daly, A. J. (2012). Teaming up: Linking

collaboration networks, collective efficacy, and student achievement. Teaching

and Teacher Education, 28(2), 251-262.

Moore, M. G. (1993). Theory for transactional distance. In D. Keegan (Ed.), Theoretical

principles of distance education (pp. 22-38). London: Routledge.

153

Moore, R. L. (2011). The effect of group composition on individual student performance

in an introductory economics course. Research in Economic Education, 42(2),

120-135.

National Academy of Sciences. (1996). National Science Education Standards.

Washington, DC: National Academy Press.

National Academy of Sciences. (2013). National Science Education Standards: An

overview. Retrieved from http://www.nap.edu/openbook.php?record_id

=4962&page=1

National Science Foundation. (2004). Science and engineering indicators: 2004.

Retrieved from http://www.nsf.gov/statistics/seind04/c7/c7s2.htm.

National Science Foundation & National Center for Science and Engineering Statistics.

(2010). Science and engineering indicators: 2010. Retrieved from http://www.

nsf.gov/statistics/seind10/c/cs1.htm

National Science Foundation & National Center for Science and Engineering Statistics.

(2012). Science and engineering indicators: 2012. Retrieved from http://www.

nsf.gov/statistics/seind12/front/fronts6.htm

National Science Teachers Association. (2012). National Science Education Standards.

Retrieved from http://www.nsta.org/publications/nses.aspx

Ni, S., & Aust, R. (2008). Examining teacher verbal immediacy and sense of classroom

community in online classes. International Journal on E-Learning, 7(3), 477-498.

Nicholas, H., & Ng, W. (2009). Engaging secondary school students in extended and

open learning supported by online technologies. Journal of Research on

Technology in Education, 41(3), 305-328.

154

Nuankhieo, P., Tsai, I., Goggins, S., & Laffey, J. (2007). Comparing the social

interaction pattern of online one-on-one peer and small group collaboration

activities. Ninth IEEE International Symposium on Multimedia 2007, 441-446.

doi:10.1109/ISM.Workshops.2007.80

Organisation for Economic Co-operation and Development. (2007). PISA 2006: Science

competencies for tomorrow’s world executive summary. Retrieved from

http://www.oecd.org/dataoecd/15/13/39725224.pdf

Ouzts, K. (2006). Sense of community in online courses. The Quarterly Review of

Distance Education, 7(3), 285-296.

Palloff, R. M. & Pratt, K. (1999). Building learning communities in cyberspace. San

Francisco: Jossey-Bass Publishers.

Palloff, R. M., & Pratt, K. (2005a). Collaborating online: Learning together in

Community. San Francisco: Jossey-Bass.

Palloff, R. M., & Pratt, K. (2005b). Learning together in community: Collaboration

online. Paper presented at the Twentieth Annual Conference on Distance

Teaching and Learning, Madison, WI. Retrieved from http://www.oakland.

k12.mi.us/Portals/0/Learning/04_1127.pdf

Palloff, R. M., & Pratt, K. (2000). Making the transition: Helping teachers to teach

online. Paper presented at EDUCAUSE 2000, Nashville, TN. Retrieved from

http://net.educause.edu/ir/library/pdf/EDU0006.pdf

Palloff, R. M., & Pratt, K. (2005b). Online learning communities revisited. Paper

presented at the Twenty-first Annual Conference on Distance Teaching and

Learning, Madison, WI. Retrieved from http://www.uwex.edu/disted/conference/

155

resource_library/proceedings/05_1901.pdf

Parveen, Q., & Batool, S. (2012). Effect of cooperative learning on achievement of

students in general science at secondary level. International Education Studies,

5(2), 154-158.

Parveen, Q., Mahmood, S. T., Mahmood, A., & Arif, M. (2011). Effect of cooperative

learning on academic achievement of 8th

grade students in the subject of social

studies. International Journal of Academic Research, 3(1), 950-954.

Peterson, S. A., Divitini, M., & Chabert, G. (2009). Sense of community among mobile

language learners: Can blogs support this? International Journal of Web Based

Communities, 5(3), 428-445.

Poore, M. (2011). Digital literacy: Human flourishing and collective intelligence in a

knowledge society. Australian Journal of Language & Literacy, 34(2), 20-26.

Prensky, M. (2009). H. Sapiens Digital: From digital immigrants and digital natives to

digital wisdom. Innovate: Journal of Online Education, 5(3).

Pugh, M. J. V., & Hart, D. (1999). Identity development and peer group formation. New

Directions for Child & Adolescent Development, 1999(84), 55-70.

Raes, A., Schellens, T., Wever, B. D., & Vanderhoven, E. (2012). Scaffolding

information problem solving in web-based collaborative inquiry. Computers &

Education, 59(1), 82-93. doi: 10.1016/j.compedu.2011.11.010

Reich, S. M. (2010). Adolescents’ sense of community on MySpace and Facebook: A

mixed-methods approach. Journal of Community Psychology, 38(6), 688-705.

Resta, P., & Laferriére, T. (2007). Technology in support of collaborative learning.

Educational Psychology Review, 19(1), 65-83.

156

Roberts, D. A. (2007). Scientific literacy/science literacy. In S. K. Abell & N. G.

Lederman (Eds.) Handbook of research on science education (pp. 729-780). New

York: Routledge.

Robinson, S., & Stubberud, H. A. (2012). Communication preferences among university

students. Academy of Educational Leadership Journal, 16(2), 105-113.

Ronsisvalle, T., & Watkins, R. (2005). Student success in online K-12 education. The

Quarterly Review of Distance Education, 6(2), 117-124.

Rossi, R. J. (1997). Serving rural youth at risk: A portrait of collaboration and

community. Journal of Education for Students Placed at Risk, 2(3), 213-227.

Rovai, A. P. (2002a). Development of an instrument to measure classroom community.

Internet and Higher Education, 5(3), 197-211.

Rovai, A. P. (2002b). Sense of community, perceived cognitive learning, and persistence

in asynchronous learning networks. Internet and Higher Education, 5(4), 319-

322.

Rovai, A. P., Baker, J. S., & Ponton, M. K. (2013). Social science research design and

statistics: A practitioner’s guide to research methods and SPSS analysis.

Chesapeake, VA: Watertree Press.

Rovai, A. P., & Jordan, H. M. (2004). Blended learning and sense of community: A

comparative analysis with traditional and fully online graduate courses.

International Review of Research in Open and Distance Learning, 5(2).

Rovai, A. P., Wighting, M. J., & Liu, J. (2005). School climate: Sense of classroom and

school communities in online and on-campus higher education courses. Quarterly

Review of Distance Education, 6(4), 361-374.

157

Rovai, A. P., Wighting, M. J., & Lucking, R. (2004). The Classroom and School

Community Inventory: Development, refinement, and validation of a self-report

measure for educational research. Internet and Higher Education, 7(4), 263-280.

Roxas, K. (2011). Creating communities: Working with refugee students in classrooms.

Democracy & Education, 19(2), 1-8.

Sadler, P. M., Coyle, H., Miller, J. L., Cook-Smith, N., Dussault, M., & Gould, R. R.

(2010). The Astronomy and Space Science Concept Inventory: Development and

validation of assessment instruments aligned with the K-12 National Science

Standards. Astronomy Education Review, 8(1), 1-26. doi:10.3847/AER2009024

Saleh, M., Lazonder, A. W., & Jong, T. (2007). Structuring collaboration in mixed-ability

groups to promote verbal interaction, learning, and motivation of average-ability

students. Contemporary Educational Psychology, 32(3), 314-331.

Sancho, M., & Cline, T. (2012). Fostering a sense of belonging and community as

children start a new school. Educational & Child Psychology, 29(1), 64-74.

Sarason, S. B. (1974). Psychological sense of community: Prospects for a community

psychology. Oxford, England: Jossey-Bass.

Schulte, L. E., Shanahan, S., Anderson, T. D., & Sides, J. (2003). Student and teacher

perceptions of their middle and high schools’ sense of community. School

Community Journal, 13(1), 7-33.

Scott, P., Asoko, H., & Leach, J. (2007). Student conceptions and conceptual learning in

science. In S. K. Abell & N. G. Lederman (Eds.) Handbook of research on

science education (pp. 31-56). New York: Routledge.

158

Shamah, D. (2011). Supporting a strong sense of purpose: Lessons from a rural

community. New Directions for Youth Development, 2011(132), 45-58.

Siemens, G. (2006). Connectivism: Learning theory or pastime of the self-amused?

Retrieved from http://www.elearnspace.org/Articles/connectivism_self-

amused.htm

Sivan, E. (1986). Motivation in social constructivist theory. Educational Psychologist,

21(3), 209-233.

Skinner, B. F. (1957). Verbal behavior: Century psychology series. East Norwalk, CT:

Appleton-Century-Crofts. doi:10.1037/11256-018

Smith, M. K. (2003). Communities of practice. Retrieved from http://valenciacollege.edu/

faculty/development/programs/tla/documents/CommunityofPractice.pdf

[Smith, N. C.?]. (n.d.). Psychometric qualities of MOSART tests. Unpublished

manuscript, Harvard-Smithsonian Center for Astrophysics, Harvard College,

Cambridge, MA.

Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H. (2004). Improving online learning:

Student perceptions of useful and challenging characteristics. The Internet and

Higher Education, 7(1), 59–70.

Songer, N. B. (2007). Digital resources versus cognitive tools: A discussion of learning

science with technology. In S. K. Abell & N. G. Lederman (Eds.) Handbook of

research on science education (pp. 471-491). New York: Routledge.

Stage, F. K., & Kinzie, J. (2009). Reform in undergraduate science, technology,

engineering, and mathematics: The classroom context. The Journal of General

Education, 58(2), 85-105.

159

Strijbos, J. W., Martens, R. L., & Jochems, W. M. G. (2004). Designing for interaction:

Six steps to designing computer-supported group-based learning. Computers &

Education, 42(4), 403-424.

Stump, G. S., Hilpert, J. C., Husman, J., Chung, W., & Kim, W. (2011). Collaborative

learning in engineering students: Gender and achievement. Journal of

Engineering Education, 100(3), 475-497.

Swan, K. (2004). Relationships between interactions and learning in online

environments. Retrieved from http://www.sloanconsortium.org/publications/

books/pdf/interactions.pdf

Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics. Boston, MA:

Pearson.

Thomas, M. J. W. (2002). Learning within incoherent structures: The space of online

discussion forums. Journal of Computer Assisted Learning, 18(3), 351-366.

Thompson, T. L., & MacDonald, C. J. (2005). Community building, emergent design and

expecting the unexpected: Creating a quality eLearning experience. The Internet

and Higher Education, 8(3), 233–249.

Top, E. (2012). Blogging as a social medium in undergraduate courses: Sense of

community best predictor of perceived learning. Internet and Higher Education,

15(1), 24-28.

Treagust, D. F. (2007). General instructional methods and strategies. In S. K. Abell & N.

G. Lederman (Eds.) Handbook of research on science education (pp. 373-391).

New York: Routledge.

160

Trust, T. (2012). Professional learning networks designed for teacher learning. Journal of

Digital Learning in Teacher Education, 28(4), 133-138.

Tutty, J. E., & Klein, J. D. (2008). Computer-mediated instruction: A comparison of

online and face-to-face collaboration. Educational Technology Research and

Development, 56(2), 101-124.

Ullmer, E. (1994). Media and learning: Are there two kinds of truth? Educational

Technology Research and Development, 42(1), 21-32.

Underwood, J., Smith, H., Luckin, R., & Fitzpatrick, G. (2008). E-science in the

classroom: Towards viability. Computers & Education, 50(2), 535-546.

Vaughan, N., & Garrison, D. R. (2005). Creating cognitive presence in a blended faculty

development community. Internet and Higher Education, 8, 1-12.

Vaughan, N., Nickle, T., Silovs, J., & Zimmer, J. (2011). Moving to their own beat:

Exploring how students use web 2.0 technologies to support group work outside

of class time. Journal of Interactive Online Learning, 10(3), 113-127.

Virginia Department of Education. (2010). Physical science. Retrieved from

http://www.doe.virginia.gov/testing/sol/standards_docs/science/2010/courses/stds

_physical_science.pdf

Virginia Department of Education. (2012). Teaching in Virginia: Performance &

evaluation. Retrieved from http://www.doe.virginia.gov/teaching/

performance_evaluation/.

Virginia Department of Education. (2011a). Testing & Standards of Learning (SOL).

Retrieved from http://www.doe.virginia.gov/testing/index.shtml.

161

Virginia Department of Education. (2011b). Virginia Standards of Learning & Common

Core State Standards. Retrieved from http://www.doe.virginia.gov/

testing/common_core/index.shtml

Vygotsky, L. (1986). Thought and language. Cambridge, MA: The MIT Press.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological

processes. Cambridge, MA: Harvard University Press.

Walker, R. M. (2013). Applied statistics: From bivariate through multivariate

techniques. Los Angeles, CA: Sage.

Wang, Q. (2010). Using online shared workspaces to support group collaborative

learning. Computers & Education, 55(3), 1270-1276.

Webb, N. M., Franke, M. L., Ing, M., Chan, A., De, T., Freund, D., & Battey, D. (2008).

The role of teacher instructional practices in student collaboration. Contemporary

Educational Psychology, 33(3), 360-381.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New

York: Cambridge University Press.

Wenger, E., White, N., & Smith, J. D. (2009). Digital habitats: Stewarding technology

for communities. Portland, OR: CPsquare.

Werner-Burke, N., Spohn, J., Spencer, J., Button, B., & Morral, M. (2012). Bridging the

disconnect: A layered approach to jump-starting engagement. Voices from the

Middle, 19(4), 45-49.

Wighting, M, Nisbet, D., & Spaulding, L. (2009). Relationships between sense of

community and academic achievement: A comparison among high school

students. The International Journal of the Humanities, 7(3), 63-72.

162

Williams, P. J. (2009). Technological literacy: A multiliteracies approach for democracy.

International Journal of Technology & Design Education, 19(3), 237-254. doi:

10.1007/s10798-007-9046-0

Winters, F. I., & Alexander, P. A. (2011). Peer collaboration: The relation of regulatory

behaviors to learning with hypermedia. Instructional Science, 39(4), 407-427. doi:

10.1007/s11251-010-9134-5

Woodworth, G. G. (1979). Bayesian full rank MANOVA/MANCOVA: An intermediate

exposition with interactive computer examples. Journal of Educational Statistics,

4(4), 357-404.

Yang, C., & Chang, Y. S. (2012). Assessing the effects of interactive blogging on student

attitudes toward peer interaction, learning motivation, and academic

achievements. Journal of Computer Assisted Learning, 28(1), 126-135.

Young, A. (2008). Structuring asynchronous discussions to incorporate learning

principles in an online class: One professor’s course analysis. MERLOT Journal

of Online Learning and Teaching, 4(2), 217-238.

Yu, A. Y., Tian, S. W., Vogel, D., & Kwok, R. C. (2010). Can learning be virtually

boosted? An investigation of online social networking impacts. Computers &

Education, 55(1), 1494-1503.

Zhao, L., Lu, Y., Wang, B., Chau, P. Y. K., & Zhang, L. (in press). Cultivating the sense

of belonging and motivating user participation in virtual communities: A social

capital perspective. International Journal of Information Management. Advance

online publication. doi: 10.1016/j.ijinfomgt.2012.02.006

163

Zhao, Y., Alvarez-Torres, M. J., Smith, B., & Tan, H. S. (2004). The non-neutrality of

technology: A theoretical analysis and empirical study of computer-mediated

communication technologies. Journal of Educational Computing Research, 30,

23-54.

Zhu, C. (2012). Student satisfaction, performance, and knowledge construction in online

collaborative learning. Educational Technology & Society, 15(1), 127-136.

164

APPENDIX A

165

APPENDIX B

Jillian

Typewritten Text

Jillian

Typewritten Text

Jillian

Typewritten Text

Jillian

Typewritten Text

Jillian

Typewritten Text

Jillian

Typewritten Text

From General Science Workbook by Pearson Staff Copyright © 1994 Pearson Education, Inc., Science Explorer Physical Science Guided Study Workbook by Prentice Hall School Publishing Staff Copyright © 2002 Prentice Hall, and Science Explorer Physical Science Unit Resources by Prentice Hall School Publishing Staff Copyright © 2004 Prentice Hall. Used by permission. All rights reserved.

166

Jillian

Typewritten Text

Jillian

Typewritten Text


167

Jillian

Typewritten Text


168

Jillian

Typewritten Text


169

Jillian

Typewritten Text


170

Jillian

Typewritten Text


171

Jillian

Typewritten Text


172

Jillian

Typewritten Text


173

Jillian

Typewritten Text


174

Jillian

Typewritten Text


175

Jillian

Typewritten Text


176

Jillian

Typewritten Text


177

Jillian

Typewritten Text


178

Jillian

Typewritten Text


179

Jillian

Typewritten Text


180

Jillian

Typewritten Text


181

Jillian

Typewritten Text


182

Jillian

Typewritten Text


183

Jillian

Typewritten Text


184

Jillian

Typewritten Text


185

Jillian

Typewritten Text


186

Jillian

Typewritten Text


187

Jillian

Typewritten Text


188

Jillian

Typewritten Text


189

Jillian

Typewritten Text


190

Jillian

Typewritten Text


191

Jillian

Typewritten Text


192

From General Science Workbook by Pearson Staff Copyright © 1994 Pearson Education,

Inc., Science Explorer Physical Science Guided Study Workbook by Prentice Hall School

Publishing Staff Copyright © 2002 Prentice Hall, and Science Explorer Physical Science

Unit Resources by Prentice Hall School Publishing Staff Copyright © 2004 Prentice Hall.

Used by permission. All rights reserved.

193

APPENDIX C

194

APPENDIX D

195

APPENDIX E

201

APPENDIX F

Classroom Community Scale (Rovai, 2002a)

Directions: Below, you will see a series of statements concerning a specific course or

program you are presently taking or have recently completed. Read each statement

carefully and place an X in the parentheses to the right of the statement that comes closest

to indicate how you feel about the course or program. You may use a pencil or pen. There

are no correct or incorrect responses. If you neither agree nor disagree with a statement or

are uncertain, place an X in the neutral (N) area. Do not spend too much time on any one

statement, but give the response that seems to describe how you feel. Please respond to

all items.

Strongly Agree Neutral Disagree Strongly

agree (SA) (A) (N) (D) disagree (SD)

1. I feel that students in this

course care about each other (SA) (A) (N) (D) (SD)

2. I feel that I am encouraged

to ask questions (SA) (A) (N) (D) (SD)

3. I feel connected to others

in this course (SA) (A) (N) (D) (SD)

4. I feel that it is hard to get

help when I have a question (SA) (A) (N) (D) (SD)

5. I do not feel a spirit

of community (SA) (A) (N) (D) (SD)

6. I feel that I receive

timely feedback (SA) (A) (N) (D) (SD)

7. I feel that this course is

like a family (SA) (A) (N) (D) (SD)

8. I feel uneasy exposing

gaps in my understanding (SA) (A) (N) (D) (SD)

9. I feel isolated in this course (SA) (A) (N) (D) (SD)

10. I feel reluctant to speak openly (SA) (A) (N) (D) (SD)

11. I trust others in this course (SA) (A) (N) (D) (SD)

12. I feel that this course

results in only modest learning (SA) (A) (N) (D) (SD)

13. I feel that I can rely on

others in this course (SA) (A) (N) (D) (SD)

14. I feel that other students

do not help me learn (SA) (A) (N) (D) (SD)

15. I feel that members of

this course depend on me (SA) (A) (N) (D) (SD)

16. I feel that I am given

ample opportunities to learn (SA) (A) (N) (D) (SD)

17. I feel uncertain about

others in this course (SA) (A) (N) (D) (SD)

202

18. I feel that my educational

needs are not being met (SA) (A) (N) (D) (SD)

19. I feel confident that

others will support me (SA) (A) (N) (D) (SD)

20. I feel that this course does

not promote a desire to learn (SA) (A) (N) (D) (SD)

203

APPENDIX G

Informed Consent Form: The Effects of Online Collaboration on Middle School

Students’ Science Literacy and Sense of Community

Your student is invited to be part of a research study that is examining the effect of online

collaboration on middle school students’ science literacy and sense of community. Your

student was selected as a possible participant because he/she may fit the criteria for this

study (i.e. a middle school student enrolled in a science course). Participation in a

research study being conducted may be helpful to increase understanding of the effect of

collaborative learning on science literacy.

This informed consent outlines the facts, implications, and consequences of the research

study. Upon reading, understanding, and signing this document, you are giving consent

for your student to participate in the research study.

Researcher:

Jillian L. Wendt, Ed.S., Doctoral Candidate, Liberty University

Inquiries:

The researcher will gladly answer any inquiries regarding the purpose and procedures of

the present study. Please send all inquiries via email to Jillian at [email protected].

Procedures: The student is being asked to complete a pre-test and post-test as part of the normal

science curriculum. The student is also being asked to complete a survey two times; once

at the beginning of the study and once at the end of the study. The length of time needed

to complete the test in class is estimated at 20- 30 minutes. The length of time needed to

complete the survey in class is estimated at 15-20 minutes. Participation is voluntary. The

researcher will take precautions to protect participant identity by not using the names of

participants or the name of the school in her results or writing. The researcher will use the

assessment results for publications and presentation purposes. Normal classroom

instruction will continue for both the experimental and control groups to which your child

may be assigned. Your child will be asked to participate in face-to-face collaborative

(group) activities or computer-mediated collaborative activities. Computer-mediated

activities will be completed through the use of Edmodo, an online educational platform.

Online collaboration will occur through the use of the Edmodo learning platform. Each

participant will be required to create a free Edmodo account and will be given a code to

access the Edmodo class. Activities for both groups will include worksheet and

discussion-type activities.

Participant Risks:The study may involve risks to the participant, which include possible

identification of the participant in relation to test or survey results. However, this risk is

minimized by only the classroom teacher and the researcher having access to student test

scores and survey results. All reported scores and results will not include student names;

rather, school-issued student identification numbers will be used. In addition, the school

204

will not be identified in published reports, but rather will be given a pseudonym. Risks of

this study are minimal and are not expected to be more than those encountered in

everyday life.

Participant Benefits: Participants may benefit from increased understanding of science literacy, including

common scientific misconceptions. Participants may also benefit from increased

understanding of the possible benefits of collaborative learning. The potential publication

of the findings of this study may prove beneficial to students, faculty, and education

administrators as they seek to proactively improve the teaching and learning process in

the high school setting.

Compensation:

Participants will not receive any financial compensation for participation in this study.

Confidentiality: The researchers will take precautions to protect participant confidentiality through the use

of school-issued identification numbers. The researcher will not identify participants by

name or identify the school in any of their writings or presentations.

The tests and surveys will be provided to participants and completed on paper.

Completed tests and surveys will be stored by the classroom teacher and/or the researcher

in a locked file cabinet. The completed tests and surveys will be stored for the duration

of three years and will then be destroyed by the researcher.

Online collaboration will occur through the use of the Edmodo learning platform. Each

participant will be given a code to access the Edmodo class. Only participants, the

classroom teacher, and the researcher will have access to the code. However, it is

conceivable that engineering staff at the web hosting company may need to access the

database for maintenance reasons. The information will be stored on this site for the

duration of three years and will then be deleted by the researchers. The researchers will

store all research documentation on a password-protected computer database on their

university computers for the duration of three years and will then delete the

documentation from the computer database. Any hard copies of the data will be stored in

a locked filing cabinet and shredded at the end of three years.

Voluntary Participation:

Participation in this study is voluntary and you or your student may withdraw at any time

without penalty. A decision to participate or not participate will not affect the student’s

relationship with Liberty University or with XXXX.

Disclosure:By signing below I acknowledge the following:

I have read and understand the description of the study and contents of this document. I

have had an opportunity to ask questions and have all my questions answered. I hereby

acknowledge the above and give my voluntary consent for my student’s participation in

205

this study. I understand that should I have any questions about this research and its

conduct, I should contact the researcher listed above.

If I have any questions about rights or this form, I should contact the researcher, Jillian

Wendt, XXXX the faculty advisor for this study, Dr. Amanda Rockinson-Szapkiw,

Liberty University, 1971 University Blvd., Lynchburg, VA 24502, (434-582-7423) or the

current IRB chair for Liberty University, Dr. Fernando Garzon, Liberty University, IRB

Review, 1971 University Blvd., Lynchburg, VA 24502.

Student Name (Print):________________________________________________

Parent/Guardian Signature:____________________________________________

Parent/Guardian Name (Print):_________________________________________

Date:_______________________

206

APPENDIX H

Assent of Child to Participate in a Research Study

What is the name of the study and who is doing the study?

The name of the study is “The Effect of Online Collaborative Learning on Middle School

Student Science Literacy and Sense of Community”. It is being completed by Jillian

Wendt, a student at Liberty University.

Why are we doing this study?

We are interested in studying how group activities that are completed using computers

might affect science literacy (science knowledge and how students are able to apply

science knowledge to real-life situations) and sense of community (how students feel they

belong in the classroom). We will compare this to group activities that are completed as

normal.

Why are we asking you to be in this study?

You are being asked to be in this research study because you are an 8th

grade physical

science student at XXXX and your school has agreed to participate.

If you agree, what will happen?

If you are in this study you will receive normal classroom instruction. You may be asked

to complete group activities as normal or you may be asked to complete group activities

using Edmodo, an online educational program. You will be asked to complete a science

pre-test (that will not count as a grade) and a survey about how you feel about your

science class. Then, at the end of the study, you will be asked to take another science test

(that will not count as a grade) and another survey about how you feel about your science

class.

Do you have to be in this study?

No, you do not have to be in this study. If you want to be in this study, then tell the

researcher. If you don’t want to, it’s OK to say no. The researcher will not be angry. You

can say yes now and change your mind later. It’s up to you.

Do you have any questions?

You can ask questions any time. You can ask now. You can ask later. You can talk to the

researcher. If you do not understand something, please ask the researcher to explain it to

you again.

Signing your name below means that you want to be in the study.

____________________________________________ ______________

Signature of Child Date

____________________________________________ ______________

Signature of Witness Date

207

Jillian Wendt, Doctoral Candidate

Dr. Amanda Rockinson-Szapkiw, Faculty Advisor, Liberty University

1971 University Blvd., Lynchburg, VA 24502

(434) 582-7423

Liberty University Institutional Review Board,

1971 University Blvd, Suite 1837, Lynchburg, VA 24502

or email at [email protected].

208

APPENDIX I

Script for Administration of the MOSART Assessment (Harvard, 2011)

Please read from the following script exactly as the script appears:

Students, please listen to and follow the directions that I am about to provide to

you exactly as they are described. You are about to begin completing an assessment on

science literacy. Science literacy means how you understand science and scientific

principles and how you can apply science to real-life situations. It also means how you

might misconceive, or misunderstand, certain science principles. This is not a graded

assessment. However, you are asked to answer all questions truthfully and to the best of

your ability. Please do not skip any questions or leave any questions blank. You will

complete this assessment on pencil and paper. Does everyone have a pencil and an

assessment paper? (Pause for student response). You will mark your answer choice by

circling the letter that corresponds with your answer. Please make sure all marks can be

easily read. When you are finished with your assessment, please turn your paper upside

down and wait quietly. Do you have any questions? (Pause for student response).

Now, let’s begin.

209

APPENDIX J

Script for Administration of the Classroom Community Scale (Rovai, 2002a) Survey

Please read from the following script exactly as the script appears:

Students, please listen to and follow the directions that I am about to provide to

you exactly as they are described. You are about to begin completing a survey on sense

of community. Sense of community means how you feel about fitting into your

classroom environment with your peers. The survey will also ask several questions about

you and your family, such as your gender and your ethnicity or race. Please answer all

questions truthfully. If you do not know the answer to a question, please skip the

question. However, you are encouraged to answer each question if possible. Do you

have any questions? (Pause for student response). Now let’s begin

210

APPENDIX K

MOSART TEACHER TRAINING

1. Visit http://www.cfa.harvard.edu/smgphp/mosart/ and sign up for an account.

2. Once you have established an account, sign in. Then, click on the “Tutorials” tab.

http://www.cfa.harvard.edu/smgphp/mosart/

211

If you have questions, please contact me by email or phone (804-938-2226). I will be

happy to help you!

212

APPENDIX L

December 12, 2012 To Whom It May Concern: This correspondence is to provide approval for Jillian Wendt to conduct her research analysis for her doctoral studies requirements with XXXXX School students at XXXXX Middle School. If there are other questions and/or concerns, please contact me at XXX-XXX-XXXX or [email protected]. Sincerely, Sharon B. Yates Director of Secondary Education & Career Technical Education

213

APPENDIX M

Jillian L. Wendt, Doctoral Candidate

Liberty University

Lynchburg, VA 24502

January 15, 2013

Dear Parents/Guardians:

I am writing to kindly request your assistance with completion of a research study for

which your child may qualify at XXXXXX. As a doctoral candidate at Liberty

University, I am completing the research study to fulfill the dissertation requirement. I

am also a full-time Biology teacher at XXXX. I understand the importance of minimal

disruption to learning as well as the importance of increasing science skills in the

classroom.

The purpose of this study will be to examine the influence of online collaborative

learning on middle school student science literacy and sense of community. Science

literacy is generally considered the ability of an individual to apply science knowledge to

current events while reducing misconceptions. With the growing advances in the

scientific world, it is important for parents, teachers, and students to be able to use

science knowledge learned in the classroom in appropriate ways in the real world. Sense

of community is how students feel they fit in and are part of the classroom community.

Understanding sense of community is important in ensuring students have a safe,

comfortable, and successful experience in the classroom.

As part of this study, your child will be asked to complete a short pre-test and a short

post-test that measures science literacy as part of the normal curriculum. Your child will

also be asked to complete a short survey at the beginning and the end of the study that

measures sense of community that is not part of the normal curriculum. Your child may

be assigned to instructional groups that use computer-mediated tools to complete

classroom activities. These computer-mediated tools will include the use of Edmodo, a

free educational platform that will host worksheet and discussion-type activities, that will

require your child to create a free Edmodo account. These activities will be equivalent to

those that are part of the normal face-to-face instruction. There are minimal risks to

participation in this study, which are not expected to exceed the risks encountered in

normal day-to-day life. Confidentiality will be maintained by the classroom teacher and

me as the researcher throughout the study. The length of the study is expected to be 9-

weeks and all results will be shared with XXXXXX to benefit your student’s educational

experience.

I would be very appreciative of your willingness to allow your child to participate in this

study. I am more than happy to answer any questions that you may have. If you choose

to allow your child to participate, please complete and return the attached form to your

child’s classroom teacher as soon as possible. You may contact me, Jillian Wendt, at

214

[email protected] at any time prior to or throughout this study. Thank you for your

consideration and assistance!

Sincerely,

Jillian L. Wendt, Doctoral Candidate

Liberty University

mailto:[email protected]

215

APPENDIX N

216

APPENDIX O

The Effect of Online Collaborative Learning on Middle School

Documents

Transcript of The Effect of Online Collaborative Learning on Middle School