Ccir efficacy studies

Conspiracy Code™: Intensive Reading

Executive SummaryResearch BaseEfficacy Study

November 2010

1

EXECUTIVE SUMMARY REPORT Conspiracy Code™: Intensive Reading Research-‐Base Documentation Efficacy Study Report

BACKGROUND AND OBJECTIVE The Conspiracy Code: Intensive Reading program was developed by Florida Virtual School (FLVS) and 360Ed, Inc. to deliver content within an online game-‐based environment to improve student reading skills. An independent research firm, Educational Research Institute of America (ERIA), was contracted to perform a literature search providing research-‐base documentation outlining best practices as identified by educational experts and academic research in the area of reading instruction as well as game-‐base technology. ERIA also executed a semester-‐long study with high school teachers and students to determine the educational efficacy of this reading improvement program compared to a control group.

METHODOLOGY Location: The research site for the efficacy study was a high school with approximate enrollment of 1,300 students (grades 9-‐12) located in the urban fringe of a large Florida city. Groups: Both a Conspiracy Code Group (90 students) and a Control Group (90 students) of struggling readers were selected from either grade 9 or 10. A randomized selection of students was not possible due to scheduling limitations. To provide quantifiable results of reading improvement, students in each group were given the same pretest and posttest to measure learning gains. Students in both groups were identified as being required to take an intensive reading course based on their Florida Comprehensive Assessment Test: Reading (FCAT) scores from the spring of 2009. Each of these students scored at a Level 1 (minimal success with grade-‐level content) or Level 2 (limited success with grade-‐level content). The Conspiracy Code students received instruction in a computer lab while the Control Group classes were conducted in a traditional classroom. Timeline: The study took place over the second semester of the 2009-‐2010 school year.

2

SUMMARY The literature search provided an abundance of research support for melding solid pedagogy with online game-‐based technology to deliver engaging explicit reading instruction to strengthen reading skills of secondary students. Cognitive neuroscience research (CNR) and related brain-‐based learning (BBL) principles serve as the pedagogical foundations for Conspiracy Code. Conspiracy Code’s gameplay design and assessment architecture are modeled around the 12 principles set forth in the Caine and Caine research of brain/mind learning. The design team also considered findings from Le Tellier‘s work on how to strengthen and build long-‐term memory. Meta-‐analysis supports the effectiveness of computer-‐assisted instruction (Liao, 1992; Bayraktar, 2001; and Means, Toyama, Murphy, Bakia, & Jones, 2009). Studies have shown positive results on using technology to teach reading and suggest that students may demonstrate greater persistence on computers, interacting with texts for more time than when using traditional reading instruction materials (National Reading Panel, 2000). Best practices and academic research are the foundation for Conspiracy Code’s explicit reading instruction relying on such expertise as Biancarosa & Snow (2006) and Graves & Avery (1997) for scaffolding instruction; Marzono for vocabulary development; and, Kellough & Kellough (2003) for tapping multiple learning modalities and intelligences to help all students perform well. The results of the efficacy study show the Conspiracy Code Group outperformed the Control Group by statistically significant margins, validating Conspiracy Code’s game-‐based instruction for reading intervention. 1. The Conspiracy Code: Intensive Reading program was successful in improving the

reading achievement of struggling grade 9 and 10 students. In a one semester study Conspiracy Code students increased the students’ reading scores from pretesting to posttesting by 6%.

2. The Conspiracy Code: Intensive Reading program was even more successful with students who scored the lowest on the pretests. The pretest/posttest scores of students scoring in the lowest half on the pretests were analyzed separately from the total group of students. The average increase for these low-‐scoring students increased by 9%, surpassing the increase of the total group of students.

3. In just one semester, Conspiracy Code: Intensive Reading students made statistically significant learning gains. Increases in pretest/posttest scores were statistically significant for comprehension, vocabulary, and total test scores. The Control Group did not make any gains over the same period; in fact, students lost ground in all three tested areas.

3

4. The Conspiracy Code students were more effective users of comprehension strategies than were the Control Group students. The results showed a statistically significant difference between the two groups favoring the Conspiracy Code students. Seventeen percent more Conspiracy Code students than Control Group students scored 60% or higher correctly on the Think Along Assessment.

5. The Conspiracy Code Group included 72% grade 9 students and only 28% grade 10 students. The Control Group included 49% grade 9 students and 51% grade 10 students which most likely accounted for higher pretest scores. The use of Conspiracy Code: Intensive Reading was the only variable to account for the fact that the Conspiracy Code students made significant gains and caught up to the higher grade level Control Group students.

6. There is significant commonality (correlation) between the Conspiracy Code Reading Assessment and the Florida Comprehensive Assessment Test (FCAT). Correlations between scores achieved by the total group of students, including both Control and Conspiracy Code Group students were quite high. All were statistically significant at the <.01 level of significance. Here are some of those correlations:

• The correlation between students’ FCAT score in spring 2009 and FCAT scores in spring 2010 was .65.

• The correlation between students’ FCAT score in spring 2009 and the pretest total score on the Conspiracy Code Reading Assessment was .61.

• The correlation between students’ FCAT score in spring 2010 and the posttest total score on the Conspiracy Code Reading Assessment was .51.

CONCLUSION Overall, Conspiracy Code: Intensive Reading was found to be based on proven pedagogy and to be very effective in providing reading support to struggling students. Results were studied for various demographic variables including gender, grade level, ethnic background, socio-‐economic status, and first or second language dominances; and, although there were some small differences, none were significant. All demographic groups showed significant learning gains using Conspiracy Code: Intensive Reading. The results provide evidence that the Conspiracy Code: Intensive Reading program was equally effective for all students. The study also revealed that the students in the Conspiracy Code Group achieved significantly better results than the Control Group students.

Conspiracy Code™: Intensive Reading Research Base

November 2010

Elizabeth Haydel, AuthorDr. Roger Farr, Reviewer

Kimberly Munroe, Reviewer

Table of Contents

Overview 3

Section 1: Effective Use of Technology 8

Section 2: Effective Instructional Strategies 15

Section 3: Meeting the Needs of All Learners 23

Section 4: Support for Teachers 27

Section 5: Effective Strategies to Teach Reading 31

Conclusion 41

Works Cited 42

Educational Research Institute of America

3

The Purpose for this Report Developing students’ reading skills and comprehension in a way that is exciting and engaging for students, as well as instructionally effective, is one of the biggest challenges educators face. Florida Virtual School (FLVS), in collaboration with 360Ed, Inc., developed Conspiracy Code: Intensive Reading to meet this challenge.

Conspiracy Code: Intensive Reading is an online course for intensive reading taught entirely through a game-based environment. In order to ensure optimal engagement and effectiveness for students, Conspiracy Code: Intensive Reading was designed to align with best practice as identified by educational experts and academic research. The purpose of this report is to share the research and theory that supports the content and instructional approaches upon which Conspiracy Code: Intensive Reading was created.

Intensive Reading is the second course created in the Conspiracy Code™ series. Conspiracy Code™ is a revolutionary line of courses that meld research-supported pedagogy and online, game-based technology. Designed to appeal to a generation of teenagers who have grown up using technology and increased their use of technology and media dramatically over the past five years (Kaiser Family Foundation—Rideout, Foehr, & Roberts, 2010), Conspiracy Code™ creates an appealing, interactive learning environment in which students can collaborate, explore, and build life-long skills around core course content.

The game in which students learn in Conspiracy Code™ takes place in the fictional locale of Coverton City, where an organization known as Conspiracy Incorporated uses nefarious means to seek world power. As students play the games, they adopt the personas of two high school students, Eddie Flash™ and Libby Whitetree™, and are teamed with a sentient computer known as B.E.N. (Bio-Electronic Navigator).

This game-based environment is designed to appeal to young people, who spend an average of 1 hour and 13 minutes per day playing video games (Kaiser Family Foundation—Rideout, Foehr, & Roberts, 2010). In addition, Conspiracy Code™ was designed to appeal to educators by offering them an opportunity to engage students in active learning that facilitates development of 21st–century skills (critical thinking, collaboration, and communication) using the efficiencies provided by technology.


The Forces behind the Development of Conspiracy Code: Intensive Reading There are three driving forces behind the development of the Conspiracy Code: Intensive Reading program. These are described more fully below, but in brief they are:

1. The ability to read and comprehend is essential for success in school, work, and life. 2. Reading continues to be a problem area in schools—especially for struggling older readers in middle school and high school. To be come successful comprehenders, these older readers must have explicit instruction in the strategies needed to comprehend—not isolated drill on separate skills. 3. Innovations in technology hold promise for helping to solve these problems, and to enable all students to read at the levels needed in the 21st century.

The Importance of Reading Reading is essential to success in school, work, and society (National Reading Panel, 2000; National Research Council, 1998). A student’s ability to make sense of grade level texts can ensure either success or failure in school, depending on the student’s ability to comprehend. Comprehension is a complex process, in which many factors play a role, including the active process of the interaction between the reader and the text, the understanding of the language and vocabulary in the text, and the learning and use of specific strategies for comprehension (National Reading Panel, 2000).

The Need for Explicit Instruction in Reading According to The National Council of Teachers of English (NCTE), 8 million U.S. students in grades 4 through 12 read below grade level (NCTE, 2010). Student test results from the National Assessment of Educational Progress (NAEP) and the ACT document the causes for concern for adolescent literacy. In 2005, fewer than 50% of high school graduates demonstrated readiness for college-level reading, as shown by their ACT scores. While NAEP reading scores of grade 4 and grade 8 students have improved slightly (33% of grade 4 students performed at or above the proficient level in 2009 compared to 29% in 1992; 32% of grade 8 students performed at or above the proficient level in 2009 compared to 29% in 1992), the scores still show cause for concern, with only slightly above one-third of students performing at or above the proficient level (National Assessment of Educational Progress, 2010). Furthermore, an analysis of the 2009 scores for reading at grade 4 and grade 8 still show racial/ethnic gaps, gender gaps,

4


and gaps by type of school (NAEP, 2010). The news at grade 12 is even more disheartening. The percentage of students performing at or above the proficient level decreased from 40% to 35% from 1992 to 2005, and performance gaps still

exist. The data seems to suggest that schools are doing a slightly better job teaching grades 4 and 8 students to decode and comprehend, but have still not solved the puzzle of how to effectively improve the comprehension of higher-level students in reading.

Meeting the needs of these struggling readers is not easy. Reading comprehension is a complex cognitive activity, involving many skills and varied strategies. While some learn to read and comprehend without explicit instruction, most students benefit from instruction in reading comprehension processes and strategies.

Whether they read or listen to texts, or do both at the same time, readers must use a variety of reading strategies, such as drawing conclusions and making inferences, in order to make sense of what they read. Readers who struggle to comprehend texts often have trouble using such strategies (Dole, Duffy, Roehler, & Pearson, 1991). For these struggling readers, explicit instruction in reading is particularly important.

The demands of the 21st century will require high levels of literacy for students to succeed in school, in work, and in a world in which increased and constant levels of communication are made possible and expected through varied, new technologies. To meet these demands, students must learn to be highly effective comprehenders of varied texts and media.

“Current difficulties in reading largely originate from rising demands for literacy, not from declining absolute levels of literacy. In a technological society, the demands for higher literacy are ever increasing, creating more grievous consequences for those who fall short.”— National Research Council, 1998, p. 1

5


The Promise of Technology for the Delivery of Effective Reading Instruction Only recently, technology was not advanced enough to deliver effective reading instruction; effective teachers using research-supported strategies had to engage with students in traditional classroom settings to make progress in developing reading skills and comprehension. Today, however, technology shows great promise in delivering effective instruction in reading.

An early meta-analysis supported the effectiveness of computer-assisted instruction (Liao, 1992) and continuing research has supported these early findings. Bayraktar (2001) conducted a meta-analysis to evaluate the impact of computer-assisted instruction and concluded that the use of computers was more effective than more traditional methods of instructional delivery (Bayraktar, 2001). The North Central Regional Educational Laboratory conducted another meta-analysis in 2003 and concluded that technology had a small, positive, and significant effect on student outcomes when compared with traditional instruction (North Central Regional Educational Laboratory, 2003). A more recent analysis conducted by the U.S. Department of Education in 2009 concluded the same; students in online conditions outperformed those in more traditional environments (Means, Toyama, Murphy, Bakia, & Jones, 2009).

According to the National Reading Panel (2000) computers can be particularly effective in supporting the teaching of reading; the panel concluded that the positive results of studies on using technology to teach reading suggest that using computer technology for reading instruction is a promising development for educators (National Reading Panel, 2000). The Panel’s findings also suggested that students may demonstrate greater persistence on computers, interacting with texts for more time than when using traditional reading instruction materials.

Furthermore, the technology environment is a native home for young people today. Technology plays a powerful role in the daily lives of young people, with a 2010 study by the Kaiser Family Foundation reporting that young people today spend more than 7 ½ hours on average per day engaged with media of different kinds (Kaiser Family Foundation—Rideout, Foehr, & Roberts, 2010). It is reasonable that students will be particularly engaged when instruction is designed to meet their needs using a blend of mediums that they choose to use almost three hours each day on average (1:29 for computer use; 1:13 for video game use) (Kaiser Family Foundation—Rideout, Foehr, & Roberts, 2010).

“The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction.”— Means, Toyama, Murphy, Bakia, & Jones, 2009, p. ix

6


The Organization of this Report This report is organized in five sections that describe the research that supports the Conspiracy Code: Intensive Reading program’s:

• Effective use of technology to deliver instruction (Section 1); • Integration of effective instructional strategies (Section 2); • Ability to meet all students’ needs (Section 3); • Support of teachers (Section 4); and • Delivery of effective strategies to teach reading (Section 5).

“Society needs educational systems that are designed for a world of possibilities and teachers who can help students survive and master that world.”— Caine & Caine, 1997b, p. 2

7


Section 1: Effective Use of Technology Technology has become an integral part of the operations of most businesses and workplaces and has become an important part of the delivery of instruction in our schools. As research from the Kaiser Family Foundation (2010) shows, the average student today uses technologies and media most of his or her waking hours. Research suggests that technology is one variable of classroom instruction that can support increased student learning and achievement. When used effectively, technology can support students’ classroom learning by providing opportunities for instructional support, continued learning, practice, and additional information. It is important to note, however, that the potential is not in the technology itself; rather it is in the potential for how various technologies can be used effectively as tools for learning, based on what we know of how students learn (Mayer, 2001). The U.S. Department of Education emphasizes this, too, in concluding that in many studies of online learning which show an advantage, the advantage may not be as a result of the medium in and of itself, but may result from the student time spent and the effective incorporation of content and pedagogical strategies (Means, Toyama, Murphy, Bakia, & Jones, 2009).

Supporting Increased Learning through Game-Based TechnologyConspiracy Code: Intensive Reading uses game-based technology effectively to immerse students into a character-based situation in which they must solve engaging problems while learning and practicing reading skills and strategies. Game-based technology uses interactive multimedia to create a learning environment for students in which the user can control many of the features of a game in order to meet a defined goal or challenge (CITEd, 2010). The use of game-based technology has been shown to be particularly effective in engaging students and supporting increased learning.

Even as the potential uses of technology in education were just becoming apparent, a meta-analysis of psychological, educational, and behavioral interventions found that computer-based instruction and simulation games designed for instruction were particularly effective (Lipsey & Wilson, 1993).

As simulations and game-based technology has been developed and employed in more technologically and instructionally sophisticated ways, research has continued to support its effectiveness. In an early investigation, Lester, Convers, Stone, Kahler, and Barlow (1997) investigated whether a bug named Herman could influence student learning. A fully expressive Herman exhibited three types of communication: spoken principle-based advice, high-level spoken advice, and spoken task-specific suggestions. Those middle-school students who engaged with versions of Herman that were fully or partially expressive outperformed children who engaged with a silent Herman or a version that

8


provided only task-specific suggestions (Lester et al., 1997). Further research has continued to demonstrate positive effects for game-based learning across the content areas and with students of different ages (Henderson, Klemes, & Eshet, 2000). Gee (2003) argues that playing games requires a specific, high level of literacy.

A number of factors of game-based technology appear to work in concert to appeal to students and stimulate learning. Animations, particularly those that are representational and realistic in appearance, have been demonstrated to be more effective than static pictures. In a meta-analysis of 26 studies, Hoffler and Leutner (2007) found an advantage across studies in favor of animation. When that animation was realistic, the effect size was large. Research by Castaneda (2008) also supported the conclusion that students engaged with computer simulations will experience positive learning effects compared to students engaged in expository instruction. In his study, Castaneda provided students with three different conditions; in one simulation condition, students could work freely, in another they were guided through the simulation, and in a third condition, students were provided with more traditional instruction without a simulation. Learning gains for students in either of the simulation conditions were higher than those who learned in a more traditional way. Rieber’s 1990 findings from a study comparing no graphics, static graphics, and animated graphics, also corroborate the findings of the benefit of animation; “animation can be used effectively to elaborate a lesson’s content” when the content is challenging and the animation is provided in conjunction with other activities (Rieber, 1990, p. 139).

Multimedia agents, or lifelike characters that serve as guides or mentors in an online environment, have also been shown to be particularly effective and engaging with students. Students with lower levels of content knowledge particularly benefit from these helpful mentors who are often embedded in game-based technology (McNamara & Shapiro, 2005).

The danger does exist that students will become so engaged in the game that they learn more about playing the game than they do the content around which the game is centered. The research of Rieber (2005) suggests that this pitfall can be avoided when the content of the game is made explicit, and students clearly understand their goals for learning. In Conspiracy Code: Intensive Reading learning goals are made explicit to students and content plays a central, not peripheral, role in the game.

9


An additional benefit of learning through game-based technology is that research suggests that the use of simulations can support increased student achievement. In an analysis of NAEP results, Archer (1998) reported that eighth graders in classes in which computers were used for simulations and applications scored higher on NAEP than other students by two-fifths of a grade level (Archer, 1998). Dani and Koenig (2008) also found that students reached higher performance levels on achievement tests when using computer-based instruction that was interactive and simulation-based, than they did as a result of traditional instruction.

Conspiracy Code Character Lineup

Mind Cleanse being played in user interface

10


Immersion in an Engaging ScenarioThe most effective instructional approaches are those that are motivating and engaging to the learners. An empirically grounded truth about learning is that students will put in the time and energy necessary to learn if they are interested in what they are learning (Eccles, Wigfield, & Schiefele, 1998; Guthrie & Humenick, 2004) and if they can relate to it (Beckwith, 1991; Chiesi, Spilich, & Voss, 1979).

In Conspiracy Code: Intensive Reading students enter into another world and interact with lifelike characters. Research supports the power of this type of learning; when an animated agent, or lifelike character who serves as the guide or teacher in a learning environment, is included, students enjoy computer-based learning more (Andre, Rist, & Muller, 1999). Moreno (2005) concluded that animated agents in computer-based learning environments who demonstrate personality characteristics and use a personal tone, addressing students directly with words like “you” and “we,” engage students more than when an impersonal, more academic tone is used. In addition to increased enjoyment and engagement, students also experience increased learning when working with an animated multimedia agent.

Students have reported more interest and ease in learning content (Lester, Stone, & Stelling, 1999) and outperformed control group students (Atkinson, 2002) when engaged with multimedia agents.

“…as in previous studies, students for whom the abstract learning activities had been embedded in meaningful and appealing fantasy contexts generally showed substantially greater motivation, involvement, and learning than those for whom the activities had not been so contextualized.”— Cordova & Lepper, 1996, p. 726

11


In their research into brain-based learning, Caine and Caine (1997a) identified 12 brain-based principles for learning. From these principles, they drew conclusions about what types of learning environments are most conducive to learning. Some of the features that make for meaningful learning include:

• An event or situation that has some aspect of a narrative or story form. • A physical context and peripheral environment that support the narrative. • Authentic social relationships. • A variety of sensory input. (Caine & Caine, 1997a, pp. 119-121)

Engaging Multiple Pathways to Learning Engaging students’ multiple pathways to learning is an essential part of effective instruction. What cognitive scientists and education researchers are beginning to understand further is that people can access information via different pathways, linguistic (words) and visual (images). A number of studies have demonstrated that students learn better when both pictures and words are used, than they do when text is provided without visuals (Levie & Lentz, 1982; Levin, Anglin, & Carney, 1987; Mayer, 2001). Dual-coding theory (see Paivio 1979, 1983, 1986) suggests that students learn better from words and pictures in combination because the words and pictures activate two different pathways to learning. “The case for multimedia rests in the premise that learners can better understand

CONSPIRACY CODE: INTENSIVE READING CONNECTIONDuring the pre-production phase of Conspiracy Code™ development, Florida Virtual Schools (FLVS) and 360Ed, Inc. conducted extensive focus group testing in order to better understand the tastes of the high-school student audience. Based upon the results, the team developed the concept, placing the game’s story in a context that the broadest possible audience will find engaging. The storyline, musical style, characters’ appearances, cartoon-style aesthetic, and gameplay mechanics were all vetted against a random sample of students. The fictional setting of a near-future city was determined to appeal to a broad, non-gender-specific audience of 15-year-olds. Furthermore, conspiracy-based fiction is especially present in pop culture media.

In addition, for the purposes of engaging students, furthering the storyline, and supporting players through the game, a diverse assortment of characters was created. Each character has a unique personality that is both engaging and functional—qualities that research demonstrates to be particularly effective in fostering student learning.

12


an explanation when it is presented in words and pictures than when it is presented in words alone.” (Mayer, 2001, p. 1)

Instruction that includes graphic organizers, multimedia, and images related to the content increase students’ learning. Nonlinguistic representations are one of the nine most effective instructional strategies identified by Marzano and colleagues (Marzano, Marzano, & Pickering, 2003).

The ability to include audio in an online environment provides a further benefit for learners. In a series of studies conducted with middle-school and college-age students, Moreno and her colleagues looked at the effects of including speech from animated agents, versus online text. Students who worked with the audio teacher demonstrated higher levels of learning on transfer tests and higher ratings of interest on surveys than those students who viewed text without audio (Moreno, Mayer, Spires, & Lester, 2001). Atkinson (2002) also found a benefit to aural explanations over text-based explanations; students in an experimental group that listened to explanations instead of reading them on the computer outperformed their peers on tests of knowledge and transfer. These findings suggest an additional benefit to the inclusion of an audio pathway for learning—along with combined text and visuals.

13


Involving Students’ Focused Attention and Peripheral PerceptionIn addition to accessing multiple pathways to learning, multimedia may be particularly effective because it involves both students’ focused attention as well as their peripheral perception. According to Caine and Caine’s brain-based principles for learning (1997a) the brain absorbs information both in the immediate focus of attention and in the periphery. As a result, all facets of the educational environment are important for learning—and students’ attention should be directed to the most salient concepts.

In a study conducted by Craig and colleagues, researchers exposed students to different conditions in a multimedia learning environment. Students who learned in sudden-onset and animation picture conditions consistently outperformed those who learned from static picture conditions, leading researchers to conclude that the sudden-onset and animation conditions improved performance by directing the student’s attention to specific elements of the pictures and connecting them with specific points in the narrative (Craig, Gholson, & Driscoll, 2002). These kinds of attentional cues in a multimedia environment can focus students’ attention appropriately, in a way that a textbook simply cannot.

In addition, the use of animated pedagogical agents in a computer-based learning environment can help to focus students’ attention appropriately. These agents can move around the screen, direct students through gazes and gestures, show emotions, and provide feedback—all to ensure that students pay attention to the information that is relevant to their task at hand (Atkinson, 2002).

14


Section 2: Effective Instructional Strategies Effective programs use instructional approaches that have been proven effective by research. A program that supports effective instruction for all students employs a purposeful organizing structure, engages students, motivates students to persist in learning, utilizes scaffolds to support learning, encourages collaboration and communication, and builds students’ academic confidence and their belief in their own self-efficacy to succeed in future academic endeavors. The Conspiracy Code: Intensive Reading program is grounded in these effective instructional approaches.

A Purposeful Organizing StructureProviding predictable routines for students supports learning. Not only does student behavior improve, but students also show greater engagement with learning and achieve at higher levels when they can predict the instructional routines in a classroom (Kern & Clemens, 2007). Predictability in the overall structure of Conspiracy Code: Intensive Reading facilitates learning for all students. Once they understand the organizing structure of the program, students know what to expect and can have confidence in their abilities to proceed from one step to the next.

In addition to predictability in the organizing structure of learning, structured support is important for students learning in game-based environments. In a review of research, deJong and van Joolingen (1998) concluded that embedded supports, such as hints, suggestions, and background knowledge were used in programs that supported student learning.

CONSPIRACY CODE: INTENSIVE READING CONNECTIONIn Conspiracy Code: Intensive Reading a purposeful organizing structure is followed and structured supports are embedded throughout.

15


Engaging StudentsAn empirically grounded truth about learning is that students will put in the time and energy necessary to learn if they are interested in what they are learning (Eccles, Wigfield, & Schiefele, 1998; Guthrie & Humenick, 2004).

Engagement is particularly crucial in teaching reading. Student engagement is a “powerful determinant of the effectiveness of any given literacy approach” (Strangman & Dalton, 2006, p. 559). Researchers have demonstrated a connection between student interest and higher cognitive recall and comprehension of text (Guthrie, Hoa, Wigfield, Tonks, Humenick, & Littles, 2007). Clearly the ability to engage students is an essential component of any effective reading program. Guthrie and Wigfield (2000) place engagement as the primary avenue through which reading instruction influences student performance. In their extensive review of the relationship between engagement and academic performance, they found that engaging reading instruction must:

• Foster student motivation (including through goal setting); • Teach and encourage use of strategies • Increase students’ conceptual knowledge; and • Foster social interaction.

Educational computer games that encourage exploration have been shown to be engaging to all students, and particularly to girls (Kinzie & Joseph, 2008). Perhaps because children see computers as both tools and toys, they see computer-based learning environments as being conducive to play and appealing in a way that is different from traditional classroom instruction (Downes, 2000).

In addition, multimedia has the potential to engage students actively, rather than as passive recipients of knowledge, encouraging greater involvement and retention of skills and knowledge. Research in cognition demonstrates that students learn and retain more when they are actively engaged in learning (Cawelti, 1999). Studies of student achievement in varied learning environments suggest that students learn best when they are actively engaged, particularly in technology-intensive environments (Rosen & Salomon, 2007). Active learning in the classroom can raise students’ awareness of the learning process, support retention of learning, develop critical thinking skills, and foster effective communication (Smith & Boyer, 1996).

“Research has demonstrated that interest is one of the motivational variables that has a powerful positive effect on individuals’ cognitive performance…”— Hidi & Boscolo, 2006, p. 146

16


Indeed, Reinking (2001) argues that a number of factors make multimedia environments more engaging to students than traditional print environments. These factors include the following:

1. Multimedia texts are interactive, so students read actively rather than passively; 2. The types of computer-based assistance available, and the ability through technology to assess and respond to students’ skills and knowledge, make computer-based reading less difficult than print; 3. Computer-based learning is often more game-based rather than ab stract; and 4. Multimedia environments offer varied options to meet students’ need for social learning.

CONSPIRACY CODE: INTENSIVE READING CONNECTIONConspiracy Code: Intensive Reading was created specifically to engage its target audience of secondary students. The storyline, music, characters’ appearances, personalities and functionality, the cartoon-style aesthetic, and gameplay mechanics were all tested with students and determined to engage a broad, non-gender-specific audience of 15-year-olds.

17


Motivating StudentsMotivation describes an internal process that enables a student to engage in a task and persist towards task completion. Humans are innately motivated to search for meaning, when learning has a purpose of reason (Caine & Caine, 1997b). The most effective instructional approaches are those that harness this natural inclination towards motivation, and are motivating and engaging to the learners. In reading, this need for motivation is particularly acute. Motivation to read has been shown to predict growth in reading comprehension (Guthrie, Hoa, Wigfield, Tonks, Humenick, & Littles, 2007).

In looking at students’ motivation for learning, Bohn, Roehrig, and Pressley (2004) concluded that teachers can employ a number of tactics to positively motivate students, including constructing lessons that are interesting, matching tasks to student abilities, and connecting reading and writing and content-area

learning. Schunk and colleagues concluded that to foster motivation, educators must “allow students to experience agency in their own learning, often by providing them with some choice and control, as well as tasks that require them to active rather than passive

learners.” (Schunk, Pintrich, & Meese, 2008, p. 327) In addition, strategy use also increases students’ motivation to learn—because successful strategy use helps students to see that they have the ability to learn (Schunk, Pintrich, & Meece, 2008).

Sustaining motivation is important. Educators must continue, and instructional programs must be designed to, sustain motivation, through active involvement, feedback, and opportunities for reflection. It is only with sustained motivation that learning takes place (Garris, Ahlers, & Driskell, 2002).

In recent studies comparing computer-based instruction with a traditional approach, computer-based learning has been shown to increase motivation and enthusiasm for learning (Abdoolatiff & Narod, 2009). Educational games have been found to be particularly effective in motivating students (Ke, 2008; Papastergiou, 2009; Tüzün, Yilmaz-Soylu, Karakus, Inal, & Kizilkaya, 2009). In Ke’s (2008) study of fifth grade students, students showed more motivation to learn when using a computer game environment than when using traditional paper-and-pencil materials for learning. Papastergiou (2009) and Tüzün and his colleagues (2009) found that both motivation and learning increased when students studied content-area concepts in game formats compared to non-game formats.

“Presenting new material, especially in small steps, allows [students] to be successful, and successful performances constitute an important means for sustaining student motivation…”— Schunk, Pintrich, & Meece, 2008, p. 305

18


The findings of research in motivation point to the effectiveness of Conspiracy Code: Intensive Reading at increasing students’ motivation to learn in reading. Providing Scaffolds to Support LearningProviding embedded scaffolds is an essential part of transitioning students from learning to independent practice and application and has been identified as “one of the most effective instructional techniques available” (Graves & Avery, 1997, p. 138). An instructional model that effectively provides scaffolding for student learning will employ a logical structure and sequence, progress from easier to more difficult tasks, provide additional information as needed (such as models or examples), and guide students, through tips, key words, and graphic organizers (Hillocks, 1993). Ultimately, scaffolds are removed and students are able to perform new skills and apply knowledge independently. The use of scaffolded instruction has been demonstrated to be one of the elements in effective in adolescent literacy interventions designed to promote reading comprehension (Biancarosa & Snow, 2006).

Computer programs designed for use in educational settings can employ various digital features to provide scaffolds that will support student learning. In a study of eighth graders working in a multimedia environment, learning increased when scaffolds for organization and higher-order thinking were built into the student program (Zydney, 2010). One way of scaffolding is to embed tips or helpful hints into a program to help students identify important information. This kind of highlighting of important ideas through hints or suggestions was identified as an effective scaffold for student learning in the research of deJong and van Joolingen (1998). Embedded questioning strategies is another research-supported scaffold shown to be effective in computer-based learning environments (see Britt & Aglinskas, 2002). Opportunities to reflect can also scaffold learning. The U.S. Department of Education identified nine studies which all supported the notion that online environments that included a tool or feature that prompted student reflection improved learner outcomes (Means, Toyama, Murphy, Bakia, & Jones, 2009). Aleven and Koedinger (2002) found that prompting students in a computer-based tutoring environment to generate self-explanations resulted in greater learning than among students in an unprompted control group (Aleven & Koedinger, 2002). Davis and Linn (2000) also found that students in a computer-based learning environment who were prompted to consider important ideas were able to apply more concepts than control-group students.

Another effective way to scaffold computer-based learning is through the use of an animated agent, a character who appears in a computer program to act as the facilitator, teacher, or guide to learners. In their research on these types of animated agents, McNamara and Shapiro (2005) found that these agents were

19


particularly helpful to students with low levels of knowledge in the content area, most likely because of the scaffolds that the agents provided, through think-alouds that modeled strategy use and explicit connections of new topics to prior learning.

Encouraging Communication and CollaborationResearch attests to the benefits of having students learn together in collaborative and cooperative groups (Cotton, 1995). Caine and Caine (1997a) identified principles from brain research with implications for educators. One factor they considered was the social nature of the brain. Their conclusion? A key principle for educators to consider in designing instruction is that learning “is profoundly influenced by the nature of the social relationships within which people find themselves.” (Caine & Caine, 1997a, p. 105) Classrooms in which students communicate and collaborate with one another have been shown to be more effective at fostering student learning. Students who participate in groups in which they depend on their group members and feel accountable to the group’s performance appear to learn more effectively.

CONSPIRACY CODE: INTENSIVE READING CONNECTIONThe Conspiracy Code: Intensive Reading program provides scaffolds to students in these ways…

20


Research and cognitive theory suggest that when students work in groups toward a common goal, they support one another, model strategies, and provide context-appropriate explanations and feedback (Slavin, 2002). Vermette (1988) asserts that educators can expect students who engage in cooperative learning to experience an increase in their:

• Understanding and application of concepts; • Use of critical thinking; • Sense of self-efficacy, or confidence in their ability to learn; • Positive attitudes towards others.

Research has also shown that cooperative learning strategies have a positive impact specifically on teaching students reading-comprehension strategies (Stevens, Slavin, & Farnish, 1991). Having peers interact over the use of reading strategies was demonstrated in research to increases student learning of strategies, encourage discussion, and increase comprehension (National Reading Panel, 2000).

CONSPIRACY CODE: INTENSIVE READING CONNECTIONIn Conspiracy Code: Intensive Reading, students engage in social learning through interactions with their peers, their teachers, and the characters who live in the game-based environment. All of these social interactions and collaborations help to facilitate increased engagement, learning, and achievement.

21


22

Building Student ConfidenceBuilding students’ confidence is another desired outcome of learning experiences—and one that can facilitate future learning.

Evidence from teachers who taught using computer-assisted learning suggested that students in an experimental group who used computers for learning showed positive attitudes and increases in their esteem. These positive changes “may indicate the potential the computer simulations programs have, especially for students with low reasoning abilities, to be successful in learning concepts and principles…” (Huppert, Lomask, & Lazarowitz, 2002, p. 819) Cordova and Lepper (1996) found that putting learning into a context for students, personalizing learning, and adding elements of choice into learning “all produced dramatic increases, not only in students’ motivation but also in their depth of engagement in learning, the amount they learned in a fixed time period, and their perceived competence and levels of aspiration.” (p. 715)

Confidence is also promoted through feedback as the students solve problems, have success, are independent in their learning, and are able to proceed at their own pace. Strategy instruction, and students’ subsequent effective use of those strategies, raises their sense of self-efficacy, or confidence in their ability to learn (Schunk, Pintrich, & Meece, 2008).

The presence of each of these elements—computer-based learning, contextual learning, a personalized environment for learning, the presence of options for student-controlled choice, and strategic supports for student success—point to the effectiveness of Conspiracy Code: Intensive Reading for building student confidence.


23

Section 3: Meeting the Needs of All Learners All teachers face the challenge of meeting the instructional needs of a wide range of students. Conspiracy Code: Intensive Reading was designed to support teachers with this challenge. Computers provide almost unprecedented opportunities to tailor instruction to the needs of the individual student. Conspiracy Code harnesses the power of multimedia to provide multiple representations of information, multiple pathways for action, and multiple means of engagement to ensure that all students are highly engaged and involved in learning. The program’s various components can help teachers meet the unique needs of varied populations of students by engaging them at appropriate developmental levels, engaging varied learning styles, allowing students to work at their own pace, and meeting the needs of 21st century learners.

Engaging Students at an Appropriate Developmental LevelThe diversity in today’s classrooms demands that teachers be knowledgeable, responsive, and well-prepared to individualize instruction to meet student needs. Research supports the idea that computer-based learning environments

foster independent learning and differentiation (Kalea, 2007). One way is through the effective delivery and analysis of formative assessment measures, as described later in this report. These types of ongoing assessments of student learning can provide teachers

with the information they need to tailor individualized instruction. Other techniques include providing tips and guidance at the point at which students need help—and then reducing scaffolded instruction when students do not need them and can complete tasks independently.

A gaming environment like that of Conspiracy Code: Intensive Reading can be particularly effective in meeting the needs of all learners because the environment can allow students to learn content at different levels of challenge (Habgood, Ainsworth, & Benford, 2005). These multiple pathways can help to facilitate engagement, as well, since students are more likely to be engaged when the level of challenge aligns with the level of their skill. Prompting and sustaining student engagement in this way creates the potential for gaming environments to positively impact student learning.

In a meta-analysis of evidence-based practices for effective online learning, the U.S. Department of Education concluded that the studies that looked at the effects of online individualized instruction found a positive effect. When learners

“Optimal learning takes place within students’ ‘zones of proximal development’ – when teachers assess students’ current understanding and teach new concepts, skills, and strategies at an according level.”— Vygotsky, 1978


24

were provided with such individualized supports as integrated tutorials; content delivered in visual, aural, and textual forms; and additional practice opportunities, their learning increased (Means, Toyama, Murphy, Bakia, & Jones, 2009). Matching the demands of instruction with the student’s ability to meet the demands is particularly important for students with learning difficulties. For their academic success, an environment that can provide instruction at the appropriate level is essential (Baker, Clark, Maier, & Viger, 2008). As is the case with all students, these students enjoy learning more when they can fully engage in the lesson (Kintsch, 1980).

Matching instructional demand with students’ levels of skill and ability is crucial to student engagement, motivation, and learning. As Caine and Caine (1997a) identified in their review of brain-based research to identify principles important for effective learning, the brain/mind learns optimally when people are challenged and encourage to take risks—but it shuts down in the face of threat, or in an educational context, when it perceives that the task or goal is impossible to meet. Engaging Varied Learning Styles“Learning modalities” refers to the primary way we take in information to help us learn. Commonly, researchers identify auditory, visual, and kinesthetic modalities (Barbe & Swassing, 1979). Howard Gardner (1983) established another way of grouping modalities. He asserts that there are several modalities or intelligences that link to our individual styles. These include verbal-linguistic (sensitivity to the meaning and order of words) and musical (sensitivity to pitch, melody, rhythm, and tone). Using programs and approaches that appeal to multiple student learning modalities and intelligences will help all students perform well (Kellough & Kellough, 2003). In evaluating the impact of incorporating new technologies in their instruction, teachers in one study cited the ability of computer-based learning environments to meet the needs of varied learning styles as a particular strength of technology; technology “helped teachers accommodate students’ varying learning styles and meet the needs of all students, both of which are essential to improving student achievement.” (Silverstein, Frechtling, & Miyaoka, 2000, p. xxi)

Research suggests that students from different ethnic and socioeconomic backgrounds have different learning modalities (Dunn, Griggs, & Price, 1993;

“…learners are different and need choice, while ensuring that they are exposed to a multiplicity of inputs. Multiple intelligences and vast ranges in diversity are…characteristic of what it means to be human.”— Caine & Caine, 1997a, p. 108

25

Hale-Benson, 1982). Sims (1988) found that underachieving African-American students, significantly more than white students, preferred sound and auditory teaching. Dunn (1995) contends that students who are underachieving and at risk benefit from visual resources that are reinforced by audio resources. Because of its inclusion of both auditory and visual components, the Conspiracy Code: Intensive Reading program is a particularly effective program for auditory and visual learners.

Research suggests that audio support while reading can improve students’ comprehension of the text (Balajthy, 2007). Following along in the text while at the same time listening to someone else read the words can help readers make sense of what they are reading. Furthermore, audio recording of texts can serve as models for reading by providing inflection, tone, voice, dialect, pacing, pausing, silence, and different voices (Baskin & Harris, 1995; Carbo, 1996). Struggling readers often have difficulty with fluency, which can make reading very laborious. Audio recordings can provide the model that these readers need to improve their fluency and ultimately their comprehension. Evidence also suggests that audio support while reading can help improve readers’ stamina and concentration (Hecker et al., 2002).

The addition of audio in computer-based learning environments has been shown to be effective. Students who listened to aural explanations outperformed their peers who read only text-based explanations in 2002 study by Atkinson. Specifically, students who listened to a human voice deliver explanations “perceived the examples presented during instruction to be less demanding. They also produced more conceptually accurate solutions on the practice problems and the near-transfer items on the posttest.” (Atkinson, 2002, p. 426)

The integration of video, audio, visual, and text in the Conspiracy Code: Intensive Reading program ensures that the content effectively reaches students with varied learning styles.

Clue with photo story



26

Providing Students with Control over Their LearningAllowing students to have control over choices and pacing in their computer-based learning environments, as is the case in Conspiracy Code: Intensive Reading, has been shown to be effective at increasing engagement and learning.

Research has repeatedly supported the idea that increasing students’ control over their learning can impact student learning, engagement, and motivation. Cordova and Lepper (1996) concluded that “students who were offered a modicum of choice over instructionally incidental aspects of the learning contexts showed greater increases in motivation and learning.” (p. 726) In their 2001 study, Yeh and Lehman found that when students could control aspects of the navigation of a computer program they showed increased learning over peers working with programs that lacked these choices for students. According to Mayer and Chandler (2001), students who worked in a multimedia environment in which they could control the pace of instruction outperformed control group students and were “better able to mentally organize the presented material into a cause-and-effect chain and to mentally relate the material with relevant prior knowledge. As predicted, this deeper level of understanding…was reflected in superior performance on tests of problem-solving transfer.” (p. 396) In a meta-analysis conducted by the U.S. Department of Education, researchers found that learning was enhanced when students could control their online actions. When computer-based learning programs included manipulations to trigger student activity, reflection, and self-monitoring, student learning increased (Means, Toyama, Murphy, Bakia, & Jones, 2009).

These findings on learner control suggest a particular advantage for students using Conspiracy Code: Intensive Reading. They are given control over the online resources and can tailor the learning experience to their unique abilities and needs.

THE CONSPIRACY CODE: INTENSIVE READING CONNECTIONIn Conspiracy Code: Intensive Reading, students control their own pacing, but they must demonstrate mastery before moving ahead. Teachers can impose “roadblocks” in the game if, through observation and evaluation, they determine that a student’s progress is unsatisfactory. These “roadblocks” require students to perform specific tasks before being allowed to resume gameplay.

27

Section 4: Support for Teachers Computer-based learning may take place online, but teachers play a crucial role in ensuring student learning. Computer-based learning program must support teachers in key ways, including facilitating teacher-student communication, providing guidance and support for individualized instruction tailored to student needs, and including ongoing formative assessments and the tools needed to analyze and evaluate results—and tailor instruction accordingly.

Facilitation for Teacher-Student CommunicationResearch suggests that students experience greater academic success when they feel connected within their learning communities (Alvermann, 2002; Bandura, 1993; Bean, 2000; Pajares, 1996; Phelps & Hanley-Maxwell, 1997). Teachers play a key role in developing this sense of community and purpose for learning. Diaz and Entonado (2009) found no important differences between the functions of teachers who taught online or in face-to-face classrooms. The conclusion? Even in an online environment, the instructor plays a significant role in developing this sense of community—which in turn leads to greater learning. Students’ sense of rapport with their teacher correlates with higher cognitive learning and participation (Frisby & Martin, 2010). Their sense of instructor immediacy—or psychological closeness and availability—also impacts students’ learning, motivation, and satisfaction (Christophel, 1990; Gorham, 1988).

However, research also suggests that a strong online course shell—with significant examples, study aids, and meaningful projects—decreases the activity level of online teachers needed for high student learning and achievement (Oliver, Osborne, & Brady, 2009). The Conspiracy Code: Intensive Reading program provides this strong structure, supporting teachers and facilitating their development of strong relationships with their students. The program fosters communication between students and teachers through a robust content delivery system named SiTi™, which stands for student interface/teacher interface. This delivery system facilitates communication between students and teachers.

Teacher-student interaction



28

The Inclusion of On-Going, Varied Assessments with FeedbackResearch has shown that student achievement increases when they receive frequent and specific feedback on their progress. Furthermore, the use of assessment information to drive instruction has been shown to have significant positive effects on learning. Ongoing and varied assessment measures provide teachers with the detailed knowledge of students’ strengths and weaknesses needed to deliver appropriate instruction.

In a research synthesis on the practices of effective teachers, Cotton (1995) concluded that effective teachers “monitor learning regularly, both formally and informally…” (p. 18) In addition, they check student progress routinely using varied assessment procedures, such as reviewing students’ work, checking homework, talking to students, administering tests, and reviewing student performance data (Cotton, 1995). Black and Williams’ 1998 findings support this conclusion. In a review of studies looking at the effects of assessment on students, they found that practices that strengthened the use of formative assessment produced “significant and often substantial learning gains.” (Black & Wiliam, 1998, p. 140) The work of the Education Trust concluded that traditionally low-performing schools that were able to reach high levels of achievement did so, in part, through the effective use of ongoing and frequent classroom assessment (Jerald, 2001).

THE CONSPIRACY CODE: INTENSIVE READING CONNECTION

In Conspiracy Code: Intensive Reading, students are assessed regularly through a mixed approach of un-graded, automatic, and graded, instructor-evaluated assessments, allowing teachers to gauge student mastery of content and concepts and development of higher-order and critical thinking skills. Assessments include such types as regular verbal interviews with teachers as well as written or project-based assignments.

29

Furthermore, ongoing classroom assessments appear to be effective for all students, but particularly for low achievers. Systematic formative assessment practices have been shown to significantly increase mildly learning disabled student’s academic achievement (Fuchs & Fuchs, 1986). In a review of effective instructional strategies for students with mild learning disabilities, Christenson, Ysseldyke, and Thurlow (1989) concluded that ongoing assessment of student progress and understanding was a critical factor for all students in any subject area. Because of its particular effectiveness for lower-performing students, the effective use of formative assessment “reduces the range of achievement while raising achievement overall.” (Black and Wiliam, 1998, p. 141)

Online environments are particularly well-suited to frequent, ongoing formative assessment. Computer-based instruction has the potential to include important, effective elements of formative assessment systems, such as frequent, consistent, timely, diplomatic, and evaluative feedback (Bischoff, 2000). On-line environments have the potential to transform assessment within classroom and allow for a continuous level of monitoring not feasible in traditional classrooms. In addition, effective computer-based assessment tools can suggest modifications and re-direct students as needed, creating a continuous cycle of learning and assessment.

“Effective online assessments should include a wide variety of clearly explained assignments on a regular basis. Feedback is also a critical component of online assessment. It must be meaningful, timely, and should be supported by a well-designed rubric when possible.”— Gaytan & McEwen, 2007, p. 129



30

CONSPIRACY CODE: INTENSIVE READING CONNECTIONSiTi, the content delivery system employed by the Conspiracy Code: Intensive Reading program, delivers benefits to teachers to facilitate ongoing, formative assessment of student performance. Through SiTi, teachers are able to easily and quickly maintain grade books, track student pacing and progress, and view student trends on both individual and aggregate levels.

The Conspiracy Code: Intensive Reading program also provides another benefit to teachers in terms of the types of assessments that are included in the program. One obstacle for many readers is that they are not always given the opportunity to move beyond literal recall of texts in the classroom because of teachers’ reliance on literal questions (Barnes, 1975). In order to improve their abilities to answer more complex questions about texts, students need to be given multiple opportunities to do so. The assessment activities embedded throughout the Conspiracy Code: Intensive Reading program were developed specifically with the intent of providing all readers, but particularly those readers who need more practice, with the opportunity to answer questions that move beyond literal recall and encourage higher-level inferences, synthesis, analysis, and evaluation.

31

Section 5: Effective Strategies to Teach Reading The primary goal of reading instruction for older students is to develop students’ ability to read and comprehend independently texts from a variety of genres and at increasing levels of difficulty. To do so, students must analyze, synthesize, and evaluate; they must make inferences, interpret, and apply what they read; and they must read extensively, for enjoyment as well as for information. An effective reading program for older readers must align with state standards, model and provide explicit strategy instruction, increase students’ vocabulary and skills for acquiring new vocabulary, build on students’ background knowledge, and integrate reading and writing. The Conspiracy Code: Intensive Reading program incorporates each of these effective strategies to teach reading.

Instruction Aligned with StandardsRecent research and experience point to the effectiveness of educational systems that define clear standards and align high-quality instruction with those standards. The Conspiracy Code: Intensive Reading program was designed to meet state and national standards. The program aligns with Florida’s Next Generation Standards and with the expectations articulated in the Common Core State Standards for English Language Arts and Literacy in History/Social Studies and Science. Unlike other computer-based learning programs that are optional, add-ons to the main program of learning, or which provide on-line, but traditional, text-based instruction, the Conspiracy Code: Intensive Reading program was designed to fully meet grade-level standards and expectations through a complete, game-based course.

In addition, the Conspiracy Code: Intensive Reading program is also consistent with standards for the knowledge and use of technology. In 2007, the International Society for Technology in Education developed a set of standards to articulate the expectations for students’ knowledge and use of technology. The Conspiracy Code: Intensive Reading program is consistent with these National Educational Technology Standards for Students (NETS-S), which, among other indicators, specify that students will “use models and simulations to explore complex systems and issues” and “evaluate and select information sources and digital tools based on the appropriateness to specific tasks” (ISTE, 2007).



32

“The past two decades of research appear to support the enthusiastic advocacy of instruction of reading strategies…The instruction of cognitive strategies improves reading comprehension in readers with a range of abilities.” — National Reading Panel, 2000, p. 4-46-47

Explicit Strategy InstructionWhen good readers read, they constantly use strategies to comprehend and make sense of what they are reading. Strategies such as setting a purpose for reading, asking and answering questions while reading, identifying main ideas and details, determining cause and effect relationships, visualizing, sequencing, monitoring for understanding and summarizing are all strategies that good readers employ, often without even being aware of doing so.

While some readers informally acquire the strategies that they need to comprehend, for many others, these strategies must be specifically taught to be learned. To be most effective, reading comprehension instruction must support students, directly and explicitly, with how to use the strategies needed to comprehend a text (Hollingsworth & Woodward, 1993; National Reading Panel, 2000). Students with reading difficulties can benefit particularly from explicit instruction in comprehension strategies (Nelson & Manset-Williamson, 2006), but poor and high achievers alike, as well as native speakers and non-native speakers of English, have been shown to benefit from explicit instruction (Alfassi, 2004; Baumann, 1984; Francis, Rivera, Lesaux, Kieffer, & Rivera, 2006a, 2006b; Klingner & Vaughn, 2004: Nokes & Dole, 2004; Rosenshine, Meister, & Chapman, 1996; Van Keer & Verhaeghe, 2005).

33

To be effective, explicit instruction must meet several criteria. Rather than simply mentioning a skill or strategy, effective instructional programs and educators guide readers as to what strategies to use, and why, when, and how to use them. Typical steps in the process include:

• Direct explanation. The teacher explains to students why the strategy helps comprehension and when to apply the strategy. • Modeling. The teacher models, or demonstrates, how to apply the strategy, usually by ‘thinking aloud’ while reading the text that the students are using. • Guided practice. The teacher guides and assists students as they learn how and when to apply the strategy. • Application. The teacher helps students practice until they apply strategies independently. (Center for the Improvement of Early Reading, 2003, p. 53)

CONSPIRACY CODE: INTENSIVE READING CONNECTIONIn Conspiracy Code: Intensive Reading, each mission students engage in provides them with the opportunity to review and practice reading skills in such areas as: • Analyzing Graphs • Analyzing Vocabulary • Creating Questions • Determining Cause and Effect • Determining the Author’s Purpose (to entertain, to inform, to persuade) • Drawing Conclusions • Making Predictions • Monitoring Understanding • Previewing the Text • Reading Different Texts Differently • Reading Selectively • Setting a Purpose for Reading • Sequencing Events • Summarizing • Understanding the Main Idea • Using Prior Knowledge • Visualizing to Comprehend Text



34

“The research on comprehension strategy teaching provides powerful evidence that most struggling readers (and many not so struggling readers) benefit enormously when we can construct lessons that help make the comprehension processes visible… Students need demonstrations of effective strategy use.” — Allington, 2001, p. 98

Conspiracy Code Full Clue

Modeling Strategy UseOne of the things that makes reading so difficult for some students to learn is that many of the processes that effective readers use are internal processes. So a challenge for teachers is to make these processes more visible to students, so that they can see, study, and emulate models of the process they are trying to learn.

Research has shown that effective teachers model reading strategies to students. In an online environment, these models can be embedded at the point of instruction, to effectively support students when they need it.

35

Vocabulary Acquisition Strategies and SkillsTexts are made up of words, and without a sufficient and growing body of vocabulary, students cannot understand the words they encounter in grade-level texts. Reading comprehension and vocabulary knowledge are strongly connected (Baumann & Kame’enui, 1991; Stahl & Fairbanks, 1986). Explicit instruction in words, therefore, directly supports text comprehension.

Incidental encounters with words have proven to be effective in helping students to acquire the thousands of words they should be learning annually (Smith, 1997). Reading appears to be the primary source of incidental vocabulary acquisition (Anderson & Nagy, 1991; Baumann & Kame’enui, 1991). Students also acquire new words by engaging in listening, speaking, and writing activities.

Research suggests, however, that while words can be learned incidentally, intentional, explicit instruction plays an important role in students’ achievement (McKeown & Beck, 1988; National Reading Panel, 2000). While incidental vocabulary acquisition is important, many students need purposeful and explicit vocabulary instruction to keep up with their peers. In fact, in its analysis of the research on vocabulary instruction, the National Reading Panel (2000) found that all students can benefit from a combination of incidental encounters and explicit instruction. According to McKeown and Beck (1988), explicit instruction is actually more effective and more efficient than incidental learning for the acquisition of specific vocabulary words. The convergence of research findings—that vocabulary knowledge is essential to comprehension, reading growth, and achievement; that the number of words students need for success is large; that incidental word learning alone is not sufficient; and, that words can effectively be taught using explicit instructional strategies—together point to a clear need for vocabulary instruction.

Important to note is that while all students will benefit from explicit vocabulary instruction, for certain student groups, this instruction is essential for their success. Research has documented the disparity between the vocabularies of socioeconomically advantaged and disadvantaged student populations (Chall, Jacobs, & Baldwin, 1990; Snow, Burns, & Griffin, 1998). Without intentional and meaningful intervention, the disparity in vocabulary knowledge between these two

“The findings on vocabulary yielded several specific implications for teaching reading. First, vocabulary should be taught both directly and indirectly. Repetition and multiple exposures to vocabulary items are important. Learning in rich contexts, incidental learning, and use of computer technology all enhance the acquisition of vocabulary.” — National Reading Panel, 2001, p. 14



36

student groups only increases over time (Baker, Simmons, & Kame’enui, 1995a). English language learners also benefit a great deal from explicit vocabulary instruction. While English-Language Learners tend to acquire social language vocabulary and skills through incidental social interactions and conversations, the acquisition of an academic vocabulary requires explicit vocabulary instruction (Francis, Rivera, Lesaux, Kieffer, & Rivera, 2006a). A third group that benefits a great deal from explicit vocabulary instruction in order not to fall further behind is struggling readers. Struggling readers make larger and faster achievement gains with the help of explicit vocabulary instruction. Struggling readers often have weaknesses in the areas of fluency, comprehension, and word analysis that make it more difficult for them to learn new words and to make sense of texts. Explicit vocabulary instruction can help them grow as readers in ways that incidental vocabulary learning cannot (Sedita, 2005).

Studying the structure of words, morphology, has also been shown to be important in developing students’ abilities to acquire vocabulary and comprehend new vocabulary when reading (Aronoff, 1994). The impact of a morphological understanding of words appears to be particularly important with older students, and equally important with native speakers, English language learners, and students in urban settings. For all of these groups, “students with greater understanding of morphology also have higher reading comprehension scores” (Kieffer & Lesaux, 2007, p. 138).

Several strategies for teaching vocabulary and vocabulary acquisition skills have been demonstrated through research to be particularly effective. These research-based strategies include:

• Direct and indirect instruction through a definitional and contextual approach (Baumann & Kame-enui, 1991; Graves, 2006; Nagy, 1988; National Reading Panel, 2000; Stahl, 1986); • Multiple and varied exposures to vocabulary words (Baumann & Kame-enui, 1991; Beck, McKeown, & Kucan, 2002; Blachowicz & Fisher, 2000; Graves, 2006; Kolich, 1988; Marzano, 2009; National Reading Panel, 2000; Stahl & Fairbanks, 1986; Stahl, 1986) which leads students to a deeper understanding of words and their multiple meanings, uses, and connotations;

“The conclusion that students with greater understanding of morphology are more successful at learning academic vocabulary and comprehending text is a strong argument for including morphology instruction in language and literacy programs, especially in urban settings.” — Kieffer & Lesaux, 2007, p. 139

37

• Frequent vocabulary instruction (Beck, McKeown, & Kucan, 2002; Marzano, 2009; National Reading Panel, 2000; Stahl & Fairbanks, 1986; Topping & Paul, 1999), which can help students improve reading comprehension and fluency; • Instruction in word morphology, or structure (Aronoff, 1994; Kieffer & Lesaux, 2007) • Use of of games to engage students in learning and allow for the review of vocabulary terms (Marzano, 2009); • Use of computer technology to support vocabulary acquisition (National Reading Panel, 2000).

CONSPIRACY CODE: INTENSIVE READING CONNECTIONIn Conspiracy Code: Intensive Reading, vocabulary instruction is provided which is designed to increase students’ reading comprehension. Specifically, students: • Apply synonyms to understand difficult vocabulary; • Use context clues to define important terms; • Focus on word parts to define key words.



38

“The review of previous material helps prepare students for new learning and creates an initial sense of self-efficacy for learning. Students are apt to believe that if they understand prerequisite materials, they will be able to learn the new material.” — Schunk, Pintrich, & Meece, 2008, p. 305

Building on Students’ Background KnowledgeResearch on cognition suggests that new information must be integrated with existing information to be deeply learned and retained. New learning takes place when we can connect new concepts and ideas to those that we already know and understand. In their principles for brain-based learning, Caine and Caine (1997a) refer to this as patterning; the brain/mind looks for patterns in the familiar and the novel, and effective instruction must give learners a chance to formulate these patterns.

Educators have known for some time that in order for learners to make sense of new information, they must be able to make connections to their prior knowledge and experiences (Afflerbach, 1986; Chiesi, Spilich, & Voss, 1979; Pressley, 2000; Snow & Sweet, 2003; Spires & Donley, 1998). That means that the new understandings and

ideas to which students are introduced in school must be both relevant and familiar enough to them that they are able to make those essential connections.

The Conspiracy Code: Intensive Reading program was designed to connect with students’ own background knowledge and previous experiences. Students are given numerous opportunities to make connections between course content, previous learning, and their personal experiences and interests. By activating students’ schema and background knowledge, the program creates a supporting structure in which students can fit new ideas, concepts, and skills.

Research attests to the benefits of making effective connections to students’ background knowledge, skills, and experiences. Students who learned from instruction designed to monitor and integrate their prior knowledge outperformed students who received traditional instruction (Dole & Smith, 1989). Additionally, connecting new information to prior knowledge has been found to positively impact the learning of students with learning disabilities (Swanson & Hoskyn, 2001). Benefits of building on student’s background knowledge, interests, and experiences include increased interest, increased motivation, increased concentration and focus, and increased learning (Williams, Papierno, Makel, & Ceci, 2004).

39

Integrating Reading and WritingIntegrating skills within and across the disciplines is particularly important in English-language arts classrooms because of the interconnectedness of reading and writing, and of speaking and listening. Cognitive scientists have demonstrated that instruction is more readily learned and retained when skills are integrated, allowing students to create pathways of learning and remembering in their minds. Researchers in reading and writing have increasingly found that reading and writing depend upon a shared base of cognitive skills and understandings, and that the relationship between the two processes is bidirection-al. That is, “the sharing of knowledge between reading and writing can go either way, from reading to writing, or from writing to reading…” (Shanahan, 2006, p. 179) Research supports the idea that connections between reading and writing are present at the narrower word level (word recognition, vocabulary, and spelling) and at the larger text level (comprehension and composition) (Berninger, Abbott, Abbott, Graham, & Richards, 2002).

In the Conspiracy Code: Intensive Reading program students integrate reading, writing, listening, and speaking as they engage in diverse activities working towards their focused goals.

Integrating reading and writing not only strengthens students’ reading and writing, it also impacts students’ word-learning skills and knowledge. According to Baker, Simmons, and Kame’enui (1995a), integrating reading and writing together also facilitates reading growth and student independence in word learning. Having students engage in writing activities has been specifically identified as an effective vocabulary teaching strategy (Klesius & Searls, 1991).

For English-Language Learners, providing multiple literacy activities in varied instructional contexts is essential. For these students, it is particularly important that instruction incorporate integrated oral, reading, and writing activities (Francis, Rivera, Lesaux, Kieffer, & Rivera, 2006a).

“Research supports the idea that writing instruction also improves reading comprehension. For example, students who are given the opportunity to write in conjunction with reading show more evidence of critical thinking about reading. Likewise, many of the skills involved in writing—such as grammar and spelling—reinforce reading skills.”— Biancarosa & Snow, 2006, p. 19



40

Conspiracy Code Writing Directions

Uncorrected Student Writing Sample

Integrating skills is particularly important in English language arts classrooms because of the interconnectedness of reading and writing, speaking and listening. Instruction is more readily learned and retained when skills are integrated, allowing students to create pathways of learning and remembering in their minds. The Conspiracy Code: Intensive Reading program provides numerous opportunities for students to integrate reading and writing to facilitate their learning.

41

Conclusion: The Research Support for Conspiracy Code: Intensive Reading ™

The Conspiracy Code: Intensive Reading program was designed, using research-based principles, to meet the challenge of teaching reading comprehension in an engaging and motivating way to adolescents.

As this report has shown, the program’s game-based technology, use of human-like animated agents, effective instructional approaches, purposeful organization, focus on engagement and motivation, and attention to the needs of diverse learners meet what research suggests are the essential elements of an effective literacy program.

The Conspiracy Code: Intensive Reading program is built upon best instructional practices as well as upon what we know about student engagement and motivation. Research suggests that use of Conspiracy Code: Intensive Reading will provide students with the skills and knowledge they need to meet the increasing literacy demands of the 21st century.



42

Works Cited

Abdoolatiff, S., & Narod, F. B. (2009). Investigating the effectiveness of computer simulations in the teaching of atomic structure and bonding. Chemistry Education in the ICT Age, 85-100.

Afflerbach, P. (1986). The influence of prior knowledge on expert readers’ importance assignment processes. In J. A. Niles & R. V. Lalik (Eds.), National reading conference yearbook. Vol. 35: Solving problems in literacy: Learners, teachers, and researchers. Rochester, NY: National Reading Conference.

Aleven, V., & Koedinger, K. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2), 147-179.

Alfassi, M. (2004) Reading to learn: Effects of combined strategy instruction on high school students. The Journal of Educational Research, 97(4), 171-184.

Allington, R. (2001). What really matters for struggling readers: Designing research-based programs. New York: Addison-Wesley.

Alvermann, D. (2002). Effective literacy instruction for adolescents: Executive summary and paper commissioned by the National Reading Conference. Journal of Literacy Research, 34, 189-208.

Anderson, R. C., & Nagy, W. E. (1991). Word meanings. In R. Barr, M. Kamil, P. Monsenthal, & D. Pearson (Eds.), Handbook of Reading Research, Vol. 2 (690-724). New York: Longman.

Andre, E., Rist, T., & Muller, J. (1999). Employing AI methods to control the behavior of animated interface agents. Applied Artificial Intelligence, 13(4-5), 415-448.

Archer, J. (1998). The link to higher scores. Education Week, 18(5), 10-19.

Aronoff, M. (1994). Morphology by itself: Stems and inflectional classes. Boston, MA: Massachusetts Institute of Technology.

Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2), 416-427.

Baker, J. A., Clark, T. P., Maier, K. S., & Viger, S. (2008). The differential influence of instructional context on the academic engagement of students with behavior problems. Teacher and Teacher Education, 24(7), 1876-1883.

43

Baker, S. K., Simmons, D. C., & Kame’enui, E. J. (1995a). Vocabulary acquisition: Curricular and instructional implications for diverse learners, Technical report no. 13. University of Oregon: National Center to Improve Tools for Educators.

Baker, S. K., Simmons, D. C., & Kame’enui, E. J. (1995b). Vocabulary acquisition: Synthesis of the research, Technical report no. 13. University of Oregon: National Center to Improve Tools for Educators.

Balajthy, E. (2007) Technology and current reading/literacy assessment strategies. The Reading Teacher, 61(3), 240-247.

Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28, 117-148.

Barbe, W. B., & Swassing, R. H. (1979). Teaching through modality strengths. Columbus, OH: Zaner-Bloser.

Barnes, D. (1975). From communication to curriculum. Harmondsworth, UK: Penguin Books.

Baskin, B. H., & Harris, K. (1995). Heard any good books lately? The case for audiobooks in the secondary classroom. Journal of Reading, 38(5), 372-376.

Baumann, J. F. (1984). The Effectiveness of a Direct Instruction Paradigm for Teaching Main Idea Comprehension. Reading Research Quarterly, 20(1), 93-115.

Baumann, J. F., & Kame’enui, E. J. (1991). Research on vocabulary instruction: Ode to Voltaire. In J. Flood, J. Jensen, D. Lapp, & J. R. Squire (Eds.), Handbook of research on teaching the English language arts (604-632). New York: Macmillan.

Bayraktar, S. (2001). A meta-analysis of the effectiveness of computer-assisted instruction in science education. Journal of Research on Technology in Education, 34(2), 173-188.

Bean, T. (2000). Reading in the content areas: Social constructivist dimensions. In M. Kamil, P. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, 631-644). Mahwah, NJ: Lawrence Erlbaum Associates.

Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: robust vocabulary instruction. New York, NY: The Guilford Press.



44

Beckwith, J. B. (1991). Approaches to learning, their context and relationship to assessment performance, Higher Education, 22(1), 17-30.

Berninger, V. W., Abbott, R. D., Abbott, S. P., Graham, S., & Richards, T. (2002). Writing and reading: Connections between language by hand and language by eye. Journal of Learning Disabilities, 35(1), 39-56.

Biancarosa, G., & Snow, C. E. (2006). Reading next—A vision for action and research in middle and high school literacy: A report to Carnegie Corporation of New York (2nd ed.). Washington, DC: Alliance for Excellent Education. Retrieved April 20, 2010 from http://www.all4ed.org/files/archive/publications/ReadingNext/ ReadingNext.pdf

Bischoff, A. (2000). The elements of effective online teaching: Overcoming the barriers to success. In K. White & B. H. Weight (Eds.), The online teaching guide: A handbook of attitudes, strategies, and techniques for the virtual classroom (57-72). Boston: Allyn and Bacon.

Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80, 139-144.

Bohn, C.M., Roehrig, A.D., & Pressley, M. (2004). The first days of school in the classrooms of two more effective and four less effective primary-grades teachers. Elementary School Journal, 104(4), 269-287.

Bransford, J., Brown, A., & Coking, R. (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Britt, M., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), 485-522.

Caine, R. N., & Caine, G. (1997a). Education on the edge of possibility. Alexandria, VA: Association for Supervision and Curriculum Development.

Caine, R. N., & Caine, G. (1997b). Unleashing the power of perceptual change: The potential of brain-based teaching. Alexandria, VA: Association for Supervision and Curriculum Development.

Carbo, M. (1996). Recorded books raise reading skills. Education Digest, 61(9), 56-59.

Castaneda, R. (2008). The impact of computer-based simulation within an instructional sequence on learner performance in a Web-based environment. PhD dissertation, Arizona State University, Tempe.

45

Cawelti, G. (Ed.). (1999). Handbook of research on improving student achievement. Arlington, VA: Educational Research Service.

Center for Implementing Technology in Education (CITEd). (2010). Learn Center and Research Center. Retrieved April 12, 2010 from http://www.cited.org/ index.aspx

Center for Implementing Technology in Education (CITEd). (2010). Learning with computer games and simulations. Retrieved April 12, 2010 from http://www.cited.org/index.aspx?page_id=143

Center for Implementing Technology in Education (CITEd). (2010). Learning with multimedia agents. Retrieved April 12, 2010 from http://www.cited.org/index.aspx?page_id=142

Center for the Improvement of Early Reading (CIERA). (2003). Put Reading First: The Research Building Blocks for Teaching Children to Read. Ann Arbor, MI. Retrieved April 20,2010 from http://www.nationalreadingpanel.org/Publications/researchread.htm

Chiesi, H., Spilich, G., & Voss, J. (1979). Acquisition of domain-related information in relation to high and low domain knowledge. Journal of Verbal Learning and Verbal Behavior, 18(3), 275-290.

Chall, J. S., Jacobs, V. A., & Baldwin, L. E. (1990). The reading crisis: Why poor children fall behind. Cambridge, MA: Harvard University Press.

Christenson, S. L., Ysseldyke, J. E., & Thurlow, M. L. (1989). Critical instructional factors for students with mild handicaps: An integrative review. Remedial and Special Education, 10(5), 21-31.

Christophel, D. M. (1990). The relationships among teacher immediacy behaviors, student motivation, and learning. Communication Education, 39(4), 323-340.

Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4), 715-730. Cotton, K. (1995). Effective schooling practices: A research synthesis 1995 update. Portland, OR: Northwest Regional Educational Laboratory. Retrieved April 20, 20101 from http://home.comcast.net/~reasoned/4410/PDFonCRM/Effective%20 School%20Prac.pdf



46

Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94(2), 428-434.

Dani, D. E., & Koenig, K. M. (2008). Technology and reform-based science education. Theory into Practice, 47(3), 204-211.

Davis, E. A., & Linn, M. C. (2000). Scaffolding students’ knowledge integration: Prompts for reflection in KIE. International Journal of Science Education, 22(8), 819-837.

deJong, T. & van Joolingen, W. R. (1998). Scientific discover learning with computer simulations of conceptual domains. Review of Educational Research, 68(2), 179-201.

Diaz, L.A., & Entonado, F. B. (2009). Are the functions of teachers in e-learning and face-to-face learning environments really different? Journal of Educational Technology and Society, 12(4), 331-343.

Dole, J. A., & Smith, E. L. (1989). Prior knowledge and learning from science text: An instructional study. In S. McCormick & J. Zutell (Eds.), Cognitive and social perspectives for literacy research and instruction. (Thirty-eighth Yearbook of the National Reading Conference, 345-352). Chicago, IL: National Reading Conference.

Downes, T. (2000). The computer as a “playable” tool. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Dunn, R., Griggs, S. A., & Price, G. E. (1993). Learning styles of Mexican-American and Anglo-American elementary school students. Journal of Multicultural Counseling and Development, 21(4), 237–247.

Dunn, R. (1995). Perceptual modalities in music listening among third-grade students. Dissertation Abstracts International Section A: Humanities and Social Sciences, 56(3-A).

Eccles, J. S., Wigfield, A., & Schiefele, U. (1998). Motivation to succeed. In W. Damon (Series Ed.) & N. Eisenberg (Ed.) Handbook of child psychology: Social, emotional, and personality development (5th ed., Vol. 3). New York: Wiley.

Francis, D. J., Rivera, M., Lesaux, N., Kieffer, M., & Rivera, H. (2006a). Practical guidelines for the education of English language learners: Research-based recommendations for instruction and academic interventions. Houston: University of Houston Center on Instruction.

47

Francis, D. J., Rivera, M., Lesaux, N., Kieffer, M., & Rivera, H. (2006b). Practical guidelines for the education of English language learners: Research-based recommendations for serving adolescent newcomers. Houston: University of Houston Center on Instruction.

Frisby, B. N., & Martin, M. M. (2010). Instructor-student and student-student rapport in the classroom. Communication Education, 59(2), 146-164.

Fuchs, L.S., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53(3), 199-208.

Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books.

Garris, R., Ahlers, R., & Driskell, J. (2002). Games, motivation and learning: A research and practice model. Simulation and Gaming, 33(4), 441-467.

Gaytan, J., & McEwen, B. C. (2007). Effective online instructional and assessment strategies. The American Journal of Distance Education, 21(3), 117-132.

Gee, J. (2003). What video games have to teach us about literacy and learning. New York: Palgrave Macmillan.

Gorham, J. (1988). The relationship between verbal immediacy behaviors and student learning. Communication Education, 37(1), 40-53.

Graves, M. F. (2006). The vocabulary book: Learning and instruction. New York: Teachers College Press.

Graves, M. F., & Avery, P. G. (1997). Scaffolding students’ reading of history. Social Studies, 88(3), 134-139.

Guthrie, J. T., Hoa, A. L. W., Wigfield, A., Tonks, S. M., Humenick, N. M., & Littles, E. (2007). Reading motivation and reading comprehension growth in the later elementary years. Contemporary Educational Psychology, 32(3), 383-313.

Guthrie, J. T., & Humenick, N. M. (2004). Motivating students to read: Evidence for classroom practices that increase reading motivation and achievement. In P. McCardle & V. Chhabra (Eds.), The voice of evidence in reading research (329-354). Baltimore, MD: Paul H Brookes Publishing.

Guthrie, J. T., & Wigfield, A. (2000). Engagement and motivation in reading. In M.L. Kamil, P.B. Mosenthal, P.D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, 403-422). Mahwah, NJ: Erlbaum.



48

Habgood, M. P. J., Ainsworth, S. E., & Benford, S. (2005). Endogenous fantasy and learning in digital games. Simulation & Gaming, 36(4), 483-498.

Hale-Benson, J. E. (1982). Black children: Their roots, culture, and learning styles. Baltimore, MD: Johns Hopkins University Press.

Hecker, L., Burns, L., Elkind, J., Elkind, K., & Katz, L. (2002). Benefits of assistive reading software for students with attention disorders. Annals of Dyslexia, 52(1), 243–272.

Henderson, L., Klemes, J., & Eshet, Y. (2000). Just playing a game? Educational simulation software and cognitive outcomes. Journal of Educational Computing Research, 22(1), 105-129.

Hillocks, G., Jr. (1993). Environments for active learning. In L. Odell (Ed.), Theory and practice in the teaching of writing (244-270). Carbondale, IL: Southern Illinois University Press.

Hoffler, T. N., & Leutner, D. (2007). Instructional animation versus static pictures: A meta-analysis. Learning and Instruction, 17(6), 722-738.

Hollingsworth, M., & Woodward, J. (1993). Integrated learning: explicit strategies and their role in problem-solving instruction for students with learning disabilities. Exceptional Children, 59(5), 444-455.

Huppert, J., Lomask, S. M., & Lazarowitz, R. (2002). Computer simulations in the high school: Students’ cognitive stages, science process skills and academic achievement in microbiology. International Journal of Science Education, 24(8), 803-821.

International Society for Technology in Education (ISTE). (2007). National Educational Technology Standards (NETS-S) and Performance Indicators for Students. Retrieved March 22, 2010, from http://www.iste.org/Content/NavigationMenu/NETS/ForStudents/ 2007Standards/NETS_for_Students_2007_Standards.pdf.

Jerald, C. D. (2001). Dispelling the myth revisited. Washington DC: Education Trust.

Kalea, H. (2007). Why use a virtual learning environment? Teaching Business and Economics, 11(2), 27-30.

Ke, F. (2008) Computer games application with alternative classroom goal structures: Cognitive, metacognitive, and affective evaluation. Educational Technology Research and Development, 56(5/6), 539-556.

49

Kellough, R. D. & Kellough, N.G. (2003). Secondary school teaching: A guide to methods and resources, 2nd Ed. Upper Saddle River, NJ: Merrill/ Prentice Hall.

Kern, L., & Clemens, N. H. (2007). Antecedent strategies to promote appropriate classroom behavior. Psychology in the Schools, 44(1), 65-75.

Kieffer, M. J., & Lesaux, N. K. (2007). Breaking down words to build meaning: Morphology, vocabulary, and reading comprehension in the urban classroom. The Reading Teacher, 61(2), 134-144.

Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why would anyone read a story anyway? Poetics, 9, 87-98.

Kinzie, M., & Joseph, D. (2008). Gender differences in game activity preferences of middle school children: Implications for educational game design. Educational Technology Research and Development, 56(5/6), 643-663.

Klesius, J., & Searls, E. (1991). Vocabulary instruction. The Journal of Educational Research, 84, 177-182.

Klingner, J., & Vaughn, S. (2004). Strategies for struggling second-language readers. In Jetton, T., & Dole, J. (Eds.), Adolescent literacy research and practice (183-209). New York: The Guilford Press.

Lester, L. C., Convers, S. A., Stone, B. A., Kahler, S. E., & Barlow, S. T. (1997). Animated pedagogical agents and problem-solving effectiveness: A large-scale empirical evaluation. In Proceedings of the eighth world conference on artificial intelligence in education (23-30). Washington, DC: IOS Press.

Lester, J. C., Stone, B., & Stelling, G. (1999). Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments. User Modeling and User-Adapted Interaction, 9(1-2), 1-44.

Levie, H. W., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educational Communication and Technology Journal, 30(4), 195-232.

Levin, J. R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D.M. Willows & H.A. Houghton (Eds.), The psychology of illustration. Vol. 1. (51-86). New York: Springer.

Liao, Y. (1992). Effects of computer-assisted instruction on cognitive outcomes: A meta-analysis. Journal of Research on Computing in Education, 24(3), 367-380.



50

Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment. American Psychologist, 48(12), 1181-1209.

Marzano, R.J. (2009). Six steps to better vocabulary instruction. Educational Leadership, 67(1), 83-84.

Marzano, R. J., Marzano, J. S., & Pickering, D. J. (2003). Classroom management that works. Alexandria, VA: ASCD.

Mayer, R. E. (2001). Multimedia learning. Cambridge: Cambridge University Press.

Mayer, R.E. & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93(2), 390-397.

McKeown, M. G., & Beck, I. L. (2004). Direct and rich vocabulary instruction. In J. F. Baumann & E. J. Kame’enui (Eds.), Vocabulary instruction: Research to practice (13-27). New York: Guilford Press.

McNamara, D. S., & Shapiro, A. M. (2005). Multimedia and hypermedia solutions for promoting metacognitive engagement, coherence, and learning. Journal of Educational Computing Research, 33(1), 1-29.

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, DC: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Policy and Program Studies Service.

Moreno, R. (2005). Multimedia learning with animated pedagogical agents. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (507-523). New York: Cambridge University Press.

Moreno, R., Mayer, R.E., Spires, H.A., & Lester, J.C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177-213.

Nagy, W. E. (1988). Teaching vocabulary to improve reading comprehension. Newark, DE: International Reading Association.

National Assessment of Educational Progress. (2010). The nation’s report card. Retrieved April 24, 2010 from http://nationsreportcard.gov/reading_2009/.

51

National Institute of Child Health and Human Development (NICHD). (2000). Report of the National Reading Panel: Reports of the subgroups. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. NIH pub. No. 00-4769. Retrieved April 20, 2010 from http://www.nichd.nih.gov/publications/nrp/report.cfm

Nelson, J. M., & Manset-Williamson, G. (2006). The impact of explicit, self-regulatory reading comprehension strategy instruction on the reading-specific self-efficacy, attributions, and affect of students with reading disabilities. Learning Disability Quarterly, 29(3), 213-230.

Nokes, J., & Dole, J. (2004). Helping adolescents read through explicit strategy instruction. In Jetton, T., & Dole, J. (Eds.), Adolescent literacy research and practice (162-182). New York: The Guilford Press.

North Central Regional Educational Laboratory. (2003). A Meta-Analysis of the Effectiveness of Teaching and Learning with Technology on Student Outcomes. Naperville, IL: North Central Regional Educational Laboratory. Retrieved October 6, 2008, from http://www.ncrel.org/tech/effects2/abstract.htm.

Oliver, K., Osborne, J., & Brady, K. (2009). What are secondary students’ expectations for teachers in virtual school environments? Distance Education, 30(1), 23-45.

Paivio, A. (1979). Imagery and verbal processes. Hillsdale, NJ: Erlbaum.

Paivio, A. (1983). Empirical case for dual coding. In J. Yuille (Ed.), Imagery, memory, and cognition: Essays in honor of Allan Paivio (307-332). Hillsdale, NJ: Erlbaum.

Paivio, A. (1986). Mental representations: A dual coding approach. New York: Oxford University Press.

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66, 543-578.

Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers and Education, 52(1), 1-12.

Phelps, L. A. & Hanley-Maxwell, C. (1997). School-to-word transitions for youth with disabilities: A review of outcomes and practices. Review of Educational Research, 67, 197-226.



52

Pressley, M. (2000). What should comprehension instruction be the instruction of? In M. Kamil, P. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3). Mahwah, NJ: Lawrence Erlbaum Associates.

Reinking, D. (2001). Multimedia and engaged reading in a digital world. In L. Verhoeven & C. Snow (Eds.), Literacy and motivation: Reading engagement in individuals and groups (195-221). Mahwah, NJ: Lawrence Erlbaum Associates.

Rideout, V. J., Foehr, U. G., & Roberts, D. F. (2010). Generation M2: Media in the Lives of 8- to 18-Year-Olds: A Kaiser Family Foundation Study. Menlo Park, CA: The Henry J. Kaiser Family Foundation. Retrieved April 12, 2010, from http://www.kff.org/entmedia/upload/8010.pdf

Rieber, L. P. (1990). Using computer-animated graphics in science instruction with children. Journal of Educational Psychology, 82(1), 135-140.

Rieber, L. (2005). Multimedia learning in games, simulations, and microworlds. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (549-567). New York: Cambridge University Press.

Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(2), 1-14. Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66(2), 181-221.

Schunk, D. H., Pintrich, P. R., & Meece, J. L. (2008). Motivation in education: Theory, research, and applications. Upper Saddle River, New Jersey: Pearson/ Merrill Prentice Hall.

Sedita, J. (2005). Effective vocabulary instruction, Insights on Learning Disabilities, 2 (1), 33-45.

Shanahan, T. (2006). Relations among oral language, reading, and writing development. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (171-183). New York: The Guilford Press.

Sims, J. (1988). Learning Styles of Black-American, Mexican-American, and White-American Third- and Fourth-Grade Students in Traditional Public Schools. Doctoral dissertation, University of Santa Barbara, Santa Barbara, CA.

53

Silverstein, G., Frechtling, J., & Miyaoka, A. (2000). Evaluation of the use of technology in Illinois public schools: Final report prepared for the Illinois State Board of Education, research division. Rockville, MD: Westat.

Slavin, R. E. (2002). Cooperative learning in elementary and secondary schools. In D. Levinson (Ed.) & P. W. Cookson & A. R. Sadovnik (Co-Eds.) Education and Sociology: An Encyclopedia (115-121). New York: RoutledgeFalmer.

Smith, C. B. (1997). Vocabulary Instruction and Reading Comprehension. Eric Digest. ED 412 506.

Smith, E. T., & Boyer, M. A. (1996). Designing in-class simulations. PS: Political Science and Politics, 29, 690-694.

Snow, C. E., Burns, M. S., & Griffin, P. (Eds.) (1998). Preventing reading difficulties in young children. Committee on the Prevention of Reading Difficulties in Young Children, Commission on Behavioral Social Sciences and Education, National Research Council. Washington, DC: National Academies Press.

Snow, C., & Sweet, A. (2003) Reading for comprehension. In A. Sweet & C. Snow (Eds.), Rethinking reading comprehension. New York: Guilford.

Spires, H., & Donley, J. (1998). Prior knowledge activation: Inducing engagement with informational texts. Journal of Educational Psychology, 90, 249-260.

Stahl, S. A. (1986). Three principles of effective vocabulary instruction. Journal of Reading, 29(7), 662-668.

Stahl, S. A., & Fairbanks, M. M. (1986). The effects of vocabulary instruction: A model-based meta-analysis. Review of Educational Research, 56, 72-110.

Stevens, R. J., Slavin, R. E., & Farnish, A. M. (1991). The effects of cooperative learning and direct instruction in reading comprehension strategies on main idea identification. Journal of Educational Psychology, 83(1), 8-16.

Strangman, & Dalton, (2006). Improving struggling readers’ comprehension through scaffolded hypertexts and other computer-based literacy program. In M. C. McKenna, L. D. Labbo, R. D. Kieffer, & D. Reinking (Eds.), International handbook of literacy and technology, Volume II (75-92). Mahwah, NJ: Lawrence Erlbaum Associations.

Swanson, H. L., & Hoskyn, M. (2001). Instructing adolescents with learning disabilities: A component and composite analysis. Learning Disabilities Research & Practice, 16(2), 109-119.



54

The National Council of Teachers of English. (2007). Adolescent literacy: A policy research brief produced by The National Council of Teachers of English. Retrieved April 19, 2010 from http://www.ncte.org/library/NCTEFiles/Resources/PolicyResearch/ AdolLitResearchBrief.pdf

The National Council of Teachers of English. (2010). Adolescent literacy. Retrieved April 19, 2010 from http://www.ncte.org/adlit.

Tüzün, H., Yilmaz-Soylu, M., Karakus, T., Inal, Y., & Kizilkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers and Education, 52(1), 68-77.

Williams, W. M., Papierno, P. B., Makel, M. C. & Ceci, S. J. (2004). Thinking like a scientist about real-world problems: The Cornell Institute for Research on Children Science Education Program. Applied Developmental Psychology, 25, 107-126.

Van Keer, H., & Verhaeghe, J. P. (2005). Effects of explicit reading strategies instruction and peer tutoring on second and fifth graders’ reading comprehension and self-efficacy perceptions. Journal of Experimental Education, 73(4), 291-329.

Vermette, P. (1988). Cooperative Grouping in the Classroom: Turning Students into Active Learners. The Social Studies, 79(6), 271-273.

Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.

Yeh, S., & Lehman, J. (2001). Effects of learner control and learning strategies on English as a foreign language (EFL) learning from interactive hypermedia lessons. Journal of Educational Multimedia and Hypermedia, 10(2), 33.

Zydney, J.M. (2010). The effect of multiple scaffolding tools on students’ understanding, consideration of different perspectives, and misconceptions of a complex problem. Computers and Education, 54(2), 360-370.

A Study of Instructional Effectiveness

Conspiracy Code™: Intensive Reading November 2010

Advisory Board: Roger Farr, President Educational Research Institute of America; Chancellor’s Professor Emeritus Indiana University

Michael Beck, President Beck Evaluation & Testing Associates, Inc. Jennifer M. Conner, Assistant Professor Indiana University

Keith Cruse, Former Managing Director Texas Assessment Program


1

Table of Contents

Overview.............................................................................................................................. 2

Data Analyses...................................................................................................................... 4

Description of the School and the Program Implementation.............................................5

Conspiracy Code Classes ............................................................................................... 6

Control Group Classes................................................................................................... 6

Description of Conspiracy Code: Intensive Reading Program ...........................................7

Description of the Assessment ........................................................................................... 8

Reading Assessment Analyses ........................................................................................... 12

Control Group/Conspiracy Code Group Comparisons................................................ 12

Conspiracy Code Group and Control Pretest/Posttest Comparisons ......................... 16

Pretest/Posttest Analyses for Higher and Lower Scoring Students............................18

Other Analyses.............................................................................................................. 21

Reading Attitude Survey Analysis: Conspiracy Code Students ....................................... 22

Think Along Assessment................................................................................................... 26

Conclusions ....................................................................................................................... 30


2

Overview

This report describes a study to determine the educational efficacy of a reading

improvement program designed for secondary school students. The program is a unique

computer based program for students whose reading skills need improvement. The

program was developed by Florida Virtual School (FLVS) and 360Ed, Inc., based on the

concept of a course taught entirely through an online game-based environment. The

result of their collaboration, Conspiracy Code™: Intensive Reading, is one course in a

line of courses that embody the use of online game-based technology.

The Educational Research Institute of America (ERIA) was contacted and asked to

conduct a study to determine the effectiveness of the program by comparing reading

improvement for students enrolled in the Conspiracy Code program to a group of

students enrolled in a control group program. A large high school located in the urban

fringe of a large Florida city was selected as the research site. Both a Conspiracy Code

Group and a Control Group of struggling readers were secured in this high school. The

Control Group students were enrolled in alternative reading programs designed for

struggling readers. All students in both groups were enrolled in either grade 9 or 10.

Students were not randomly assigned to either the control classes or the Conspiracy

Code classes. Course scheduling limitations prevented such random assignment.

The study took place over the second semester of the 2009-2010 school year. Pretests

and posttests were administered to both Control Group students and Conspiracy Code

students. The pretests were used to determine students’ pre-treatment reading levels.


3

Research Questions

The following five questions guided the design of the study and the data analyses:

1. Is the Conspiracy Code: Intensive Reading program effective in improving

the reading comprehension and reading vocabulary skills of struggling readers in

grades 9 and 10; and, is it more effective than an alternate program used by

students in a control group?

2. Is the Conspiracy Code: Intensive Reading program equally effective in

improving the reading comprehension and reading vocabulary skills of the lowest

performing students to the same extent as higher performing students?


students’ attitudes toward reading?


student’s reading/thinking strategies; and, is it more effective than an alternate

program used by students in a control group?

5. Were any achievement gains made by Conspiracy Code: Intensive Reading

students influenced by demographic variables including gender, grade level,

ethnic background, socio-economic status, or first or second language

dominance?


4

Data Analyses

A Reading Assessment was developed to compare pretest and posttest scores. The

development of this assessment and pertinent reliability and validity test statistics are

presented later in this report. The test produced three test scores Reading

Comprehension, Reading Vocabulary, and Total Reading. A validity analysis of the

Reading Assessment was conducted by comparing correlations of the Reading

Assessment scores for both the Conspiracy Code Group and the Control Group students

with the students’ Florida Comprehensive Assessment Test® Scores: Reading (FCAT).

The following statistical analyses were used to compare the Conspiracy Code Group and

the Control Group students’ pretest scores to posttest scores:

• Independent sample t-tests were computed to determine if there were significant

differences between the Conspiracy Code Group and the Control Group at

pretesting and posttesting.

• Paired comparison t-tests were used to compare the pretest and posttest scores

for the Conspiracy Code Group and the Control Group.

• The Conspiracy Code Group was split into two groups based on pretest scores.

Paired comparison t-tests were used with the group that scored highest and the

group that scored lowest on the pretest.

• Descriptive statistics were used to compare pretest/posttest percent of students

scoring at low, medium, and high percent correct scores.

An effect-size analysis was computed for each of the independent sample and paired t-

test paired comparisons. Cohen’s d statistic was used to determine the effect size. This

statistic provides an indication of the strength of the effect of the treatment regardless of

the statistical significance. Cohen’s d statistic is interpreted as follows:

.2 = small effect .5 = medium effect .8 = large effect


5

Description of the School and the Program Implementation

Both the Control Group and the Conspiracy Code Group students were selected from the

same high school. After learning about the Conspiracy Code: Intensive Reading

program, the school agreed to participate in the program. The school was located in the

urban fringe of a large city and enrolled approximately 1,300 students in grades 9 to 12.

Forty-eight percent of the students in the school were enrolled in free or reduced lunch

programs. The racial mix of the school included 50% Hispanic, 29% Caucasian, 15%

African American, and 6% other.

The pilot for the Conspiracy Code: Intensive Reading course began in January 2010 and

continued until the end of the school year, June 2010. The students in the study were

those ninth and tenth grade students who scored at a level 1 or 2 on the Florida

Comprehensive Assessment Test: Reading (FCAT) the previous year. Students in Florida

who score at those levels are required to take an intensive reading course. The five score

levels on the FCAT are defined as follows:

Level 5: Successful with the most challenging grade-level content

Level 4: Mostly successful with challenging grade-level content

Level 3: Partly successful with grade-level content

Level 2: Limited success with grade-level content

Level 1: Minimal success with grade-level content

In this study, there were approximately 300 grade 9 and 10 students with Level 1 and

Level 2 scores on the FCAT administered in the spring of 2009. Of these 300 students

most of those students with the lower scores on the FCAT were placed in the Conspiracy

Code program. The Conspiracy Code Group totaled 90 students. The remaining students

out of the total of 300 had generally higher FCAT scores than the group selected for the

Conspiracy Code program. These students were placed in other reading classes using

different intensive reading programs. Of these approximately 210 students roughly 90

were selected as the Control Group.


6

Conspiracy Code Classes

Two FLVS literacy coaches were the teachers of record for the Conspiracy Code classes.

Four online FLVS reading teachers acted as teacher assistants and they worked with the

literacy coaches to assist in the implementation of the course. A reading teacher from

the high school filled the role of lab facilitator at the high school.

The Conspiracy Code classes were conducted in a computer lab which included 30

computers that met the technical specifications for the Conspiracy Code game. Classes

were held 5 days a week for 90 minute class periods except Wednesdays when classes

were 75 minutes long.

Pretests for the Conspiracy Code students were administered at the end of January and

posttests were administered at the beginning of June. The assessments were

administered online during regular Conspiracy Code class periods.

Students in the Conspiracy Code classes were introduced to computer learning by means

of a student orientation program provided by Florida Virtual School. The orientation is

described by the Florida Virtual School as follows:

The Florida Virtual School New Student Orientation is designed for students new to FLVS and virtual learning. It is designed to assess students’ readiness for online learning and to help the students acquire the skills needed for success in the online-education environment. It includes a basic computer skills diagnostic test, a module on time management and a module on internet safety.

Control Group Classes

The Control Group classes were conducted in traditional classrooms with an average of

20 students in each class. Teachers were all experienced in the teaching of reading. For

the control group the pretests and posttests were administered via paper/pencil during a

regular intensive reading class period by the reading teacher of record at the high

school.

Students in both the Conspiracy Code classes and the Control Group classes were

assigned course grades at the end of the program.


7

Description of Conspiracy Code: Intensive Reading Program

The Conspiracy Code: Intensive Reading program is designed to seamlessly meld

research-supported pedagogy and online, game-based technology. Designed to appeal to

a generation of teenagers who have grown up using technology and have increased their

use of technology and media dramatically over the past five years, Conspiracy Code

creates an appealing, interactive learning environment in which students can

collaborate, explore, and build life-long skills around core course content.

Cognitive neuroscience research (CNR) and related brain-based learning (BBL)

principles serve as the pedagogical foundations for Conspiracy Code. Conspiracy

Code's gameplay design and assessment architecture are modeled around the

12 principles set forth in the Caine and Caine research of brain/mind learning. The

design team also considered findings from Le Tellier‘s work on how to strengthen and

build long-term memory. To optimize and reinforce learning, Conspiracy Code

stimulates the active processing of experiences through questioning, feedback, focusing

students’ attention and stimulating their peripheral perceptions at the same time.

Students are continually required to identify characteristics, see relationships, analyze

situations, make critical decisions, reflect, and communicate their understanding in

unique ways.

The game in which students learn in Conspiracy Code takes place in the fictional locale

of Coverton City, where an organization known as Conspiracy Incorporated uses

nefarious means to seek world power. As students play the games, they adopt the

personas of two high school students, Eddie Flash™ and Libby Whitetree™, and are

teamed with a sentient computer known as B.E.N. (Bio-Electronic Navigator).

This game-based environment is designed to appeal to adolescents, who spend an

average of 1 hour and 13 minutes per day playing video games. Conspiracy Code was

designed to appeal to educators by offering them an opportunity to engage students in

active learning facilitating development of 21st–century skills (critical thinking,

collaboration, and communication) using the efficiencies provided by technology.


8

Description of the Assessment

The Conspiracy Code Reading Assessment was developed by ERIA staff and

Conspiracy Code staff. The assessment consists of 3 reading selections, 41 multiple

choice comprehension items, and 20 multiple choice vocabulary items.

The reading comprehension passages were written to provide students with reading

material they might find in real world situations. The passage length and readability

level of each passage are:

Passage 1 446 words 6.2 readability level Passage 2 396 words 8.9 readability level Passage 3 707 words 9.8 readability level Average 516 words 8.3 readability level

The comprehension test items were written to assess the major reading comprehension

skills listed on many reading standards analyses. The number of test items for each

comprehension category and for each reading selection is shown in Table 1.

Table 1 Reading Comprehension Test Item Categorization*

Comprehension Skills Passage 1 Passage 2 Passage 3

Total Category Items

Setting Purpose 11 16 29, 31, 33 5 Prior Knowledge 8, 14 27, 28 30 5 Previewing 10 21 32 3 Predicting 7 24 -- 2 Visualizing 6 22 34, 41 4 Questioning 2 23 35 3 Vocabulary 1, 12 20, 26 36 5 Summarizing 3, 9 18, 25 37 5 Monitoring Understanding

5, 13 17 38 4

Author’s Purpose -- 15 39 2 Words in Context 4 19 40 3 TOTAL 41 *Table data represents the numbering of items on the test.


9

The 20 vocabulary words were assessed by presenting a sentence which included a

blank. Students were asked to select from four choices the word that made sense in each

sentence. Care was taken to develop distractor choices, words that fit the syntax of the

sentence and that relate to the general meaning of the sentence. Words were chosen for

the test items that were identified by researchers as level 2 words for beginning

secondary school students. Level 2 words are beyond the words used in everyday

language which are categorized as level 1 words. Level 3 words are those that relate only

to a specific subject matter area and are much less common than level 2 words.

Evidence of the validity of the reading assessment developed for the study is provided in

Table 2 which shows the correlation between 6 different assessments. The first two

assessments are the students’ scores on the FCAT Reading1 test administered in the

spring of 2009 and the FCAT Reading test administered in the spring of 2010. The third

and fourth assessments are the FAIR2 Reading Comprehension tests administered to all

of the students in the study in both January 2010 and May 2010. The final two

assessments are the Reading Assessment Pretest and the Reading Assessment Posttest

used in the study.

1 FCAT is the Florida Comprehensive Assessment Test administered annually to Florida students. 2 FAIR is the Florida Assessment for Instruction in Reading is given periodically throughout the school year to help teachers prepare students for the end of year FCAT assessments.


10

The correlations in Table 2 indicate that all of the correlations3 were statistically

significant. It is expected that the FCAT 2009 scores and the FCAT 2010 scores would

be highly correlated. However, it is important to note that the FCAT 2009 and the

Reading Comprehension Pretest are correlated at almost the same level as the FCAT

2009 and the FCAT 2010. In fact, all of the correlations are at fairly high levels and

indicate there is a significant similarity between each of the assessments.

Table 2 Correlations* between FCAT Assessments, FAIR Assessments, Program

Pretests, and Program Posttests based on Scores for both Conspiracy Code and Control Group Students

FC

AT

Sp

rin

g 20

09

FC

AT

Sp

rin

g 20

10

FA

IR C

omp

reh

ensi

on

Jan

. 20

10

FA

IR C

omp

reh

ensi

on

May

20

10

Pre

test

Rea

din

g A

sses

smen

t

Pos

ttes

t R

ead

ing

Ass

essm

ent

FCAT Spring 2009 1.0

FCAT Spring 2010 .65 1.0

FAIR Comprehension Jan. 2010 .56 .60 1.0

FAIR Comprehension May 2010 .53 .63 .58 1.0

Pretest Reading Assessment .61 .48 .58 .60 1.0

Posttest Reading Assessment .38 .51 .46 .55 .60 1.0

*All correlations are statistically significant at the 0.01 level.

3 All correlations in Table 2 were corrected for attenuation.


11

The test statistics for both the Reading Program Pretests and Posttests for both the

Conspiracy Code students and the Control students are provided in Table 3. The pretest

and posttest reliability indexes are high and indicate that the tests are providing reliable

results for each group of students.

Table 3 Raw Score Reading Test Results Reading Pretests and Posttests

Conspiracy Code and Control Group Students

Test Number of Items

Mean Score

Standard Deviation KR 20** SEm*

Pretest Conspiracy Code Group

Comprehension 41 19.7 7.0 .83 3.62

Vocabulary 20 7.5 4.0 .77 1.63

Total Score 61 27.1 9.9 .88 3.43

Posttest Conspiracy Code Group

Comprehension 41 22.0 7.9 .88 2.74

Vocabulary 20 9.8 4.7 .83 1.94

Total Score 61 31.9 11.8 .92 3.34

Pretest Control Group

Comprehension 41 25.4 7.1 .86 2.66

Vocabulary 20 11.3 6.1 .93 1.61

Total Score 61 36.7 12.0 .93 3.17

Posttest Control Group

Comprehension 41 24.3 7.9 .88 2.74

Vocabulary 20 10.7 5.6 .90 1.77

Total Score 61 35.1 12.4 .93 3.28 *SEm stands for Standard Error of Measurement **KR 20 stands for Kuder Richardson 20 which is a commonly used reliability index


12

Reading Assessment Analyses

All of the raw scores for the pretests and posttests were converted to standard scores for

analysis. The purpose for this conversion was to produce scores that had equal standard

deviations. Both Control Group and Conspiracy Code students’ pretest scores were

combined to develop the standard score scale. The mean of the standard score

developed for the study was 300 and the standard deviation was 50. Thus a student

whose score was exactly the same as the average score for the total group would be

assigned a score of 300 on the standard score scale. A student whose raw score was

exactly one standard deviation above the mean would be assigned a standard score of

350. Conversion tables were developed to convert each student's pretest and posttest

scores from a raw score to a standard score.

Control Group/Conspiracy Code Group Comparisons

Since the Control Group students and the Conspiracy Code Group students were not

randomly assigned to either group, it was necessary to compare the two groups on the

pretests to determine if the groups differed in their performance on the pretests. An

independent sample t-test was used to analyze the pretest scores. Table 4 shows that the

Control Group scored at a statistically significant higher level on the pretest than the

Conspiracy Code Group. The Control Group scored statistically significantly higher than

the Conspiracy Code Group on the Comprehension, Vocabulary, and Total Test pretests.

The statistical significance for all three comparisons was (≤.0001) indicating a difference

that would occur by chance less than once out of 10,000 repetitions. In addition the

effect sizes were all large. The analysis indicates that the Control Group was achieving at

a much higher level at the outset of the study than the Conspiracy Code Group.


13

As reported above, it was discovered that the assignment of students to either group

revealed that the school personnel making the assignments assumed that the Conspiracy

Code Group was to include the least able readers. Key demographic data indicate fairly

large differences between the two groups. The Control Group included 51% grade 10

students and 49% grade 9 students while the Conspiracy Code Group included only 28%

grade 10 students and 72% grade 9 students. The Control Group included 73% of the

students enrolled in free or reduced price lunch programs while the Conspiracy Code

included 83% of the students enrolled in such programs. Thus, it was clear that the

Control Group included a higher socio-economic group and a higher grade level group.

In addition, the Conspiracy Code Group students were selected to be in that group

because they scored lower on the 2009 FCAT reading assessments. The pretest score

differences between the two groups reflected these differences.

Table 4 Independent Sample Comparison t-test Results

Comparing Students’ Pretest Standard Scores for Conspiracy Code (N=78) and Control Group (N=70)

Tes

t

Gro

up

Mea

n S

tan

da

rd

Sco

re

Sta

nd

ard

D

evia

tion

t-te

st

Sig

nif

ica

nce

Eff

ect

Siz

e

Comprehension Conspiracy Code

282 46

Comprehension Control 320 47 4.966 ≤.0001 .80

Vocabulary Conspiracy Code 282 38

Vocabulary Control 320 55 4.857 ≤.0001 .80

Total Conspiracy Code

281 42

Total Control 322 50 5.428 ≤.0001 .90


14

Table 5 provides the same analysis for the posttest scores. The comparison of the

Control Group and the Conspiracy Code Group on the posttests showed statistically

significant differences for only the Comprehension Test score with a statistical

significance of ≤.03 indicating a difference that would occur by chance less than three

times out of 100 repetitions. The effect size was small.

While the Control Group mean scores were statistically significantly higher than the

Conspiracy Code Group at the time of the pretest, the advantage for the Control Group

almost totally disappeared by the time of posttesting. The Conspiracy Code Group made

greater gains over the course of the year than did the Control Group.

Table 5 Independent Sample Comparison t-test Results

Comparing Students’ Posttest Standard Scores for Conspiracy Code (N=78) and Control Group (N=70)

Tes

t

Gro

up

Mea

n S

tan

da

rd

Sco

re

SD

t-te

st

Sig

nif

ica

nce

Eff

ect

Siz

e

Comprehension Conspiracy Code

294 53

Comprehension Control 313 54 2.187 ≤.03 .35

Vocabulary Conspiracy Code 301 42

Vocabulary Control 309 51 .978 Non-Sig. --

Total Conspiracy Code

297 49

Total Control 312 53 1.846 Non-Sig. --


15

Figure 1 shows the advantage for the Control Group at pretesting. Figure 2 shows how

that advantage disappeared by posttesting. In fact, as will be shown in Table 6, the

Control Group students made no significant gains from pretesting to posttesting while

the Conspiracy Code Group students made statistically significant gains on all three test

comparisons.

Figure 1 Comparison of Pretest Scores of

Control Group and Conspiracy Code Students

Figure 2 Comparison of Posttest Scores of

Control Group and Conspiracy Code Students


16

Conspiracy Code Group and Control Pretest/Posttest Comparisons

Table 6 shows that the Conspiracy Code students made statistically significant gains on

all of three comparisons. Data results for the Control Group students revealed no

statistically significant gains from pretesting to posttesting for all 3 comparisons. In fact,

the Control Group students declined slightly, but not statistically significantly, from

pretesting to posttesting.

Table 6 Pretest to Posttest Comparison for

Control Group Students and Conspiracy Code Students

Tes

t

Tes

t F

orm

Nu

mbe

r of

S

tud

ents

Mea

n

Sta

nd

ard

S

core

SD

t-te

st

Sig

nif

ica

nce

Eff

ect

Siz

e

Control Group Students

Total Pretest 70 322 50

Total Posttest 70 312 53 -1.391 Non-Sig. --

Comprehension Pretest 70 320 47

Comprehension Posttest 70 313 54 -1.119 Non-Sig. --

Vocabulary Pretest 70 320 55

Vocabulary Posttest 70 309 51 -1.459 Non-Sig. --

Conspiracy Group Students


Total Posttest 78 297 49 3.647 ≤.0001 .35


Comprehension Posttest 78 294 53 2.466 ≤.02 .31


Vocabulary Posttest 78 301 42 3.889 ≤.0001 .51


17

Figure 3 shows the Conspiracy Code students pretest to posttest for the percentage of

students scoring below 60% correct and 60% correct or higher. There was a decline of

13% at the lowest achievement level and an increase of 13% at the highest achievement

level.

Figure 3 Comparison of Pretest/Posttest Total Percentage Correct Scores of

Conspiracy Code Students


18

Figure 4 shows the Control Group students pretest to posttest for the percentage of

students scoring below 60% correct and 60% correct or higher. There was an increase of

12% at the lowest achievement level and a decrease of 12% at the highest achievement

level.


Control Group Students

Pretest/Posttest Analyses for Higher and Lower Scoring Students

Comparisons were next made between the lowest scoring pretest Conspiracy Code

students and the highest scoring pretest Conspiracy Code students. The total group of

78 Conspiracy Code students was ranked from high to low on the basis of their pretest

scores. The lowest 39 students were considered the lower scoring pretest students and

the highest 39 were considered the higher scoring pretest students.


19

The results of the pretest/posttest analyses for the lower and higher scoring pretest

Conspiracy Code Groups are shown in Table 7.

Table 7 Pretest to Posttest Comparison for

Low Scoring Pretest and High Scoring Pretest Conspiracy Code Students

Tes

t

Tes

t F

orm

Nu

mb

er o

f S

tud

ents

Mea

n

Sta

nd

ard

S

core

SD

t-te

st

Sig

nif

ica

nce

Eff

ect

Siz

e

Lower Scoring Group


Total Posttest 39 269 41 3.361 ≤.002 .74


Comprehension Posttest 39 264 45 2.620 ≤.01 .50



Higher Scoring Group


Total Posttest 39 324 41 1.666 Non-Sig. --


Comprehension Posttest 39 324 43 .794 Non-Sig. --




20

Figures 5 and 6 provide a pretest-to-posttest comparison of the percentage of lower and

higher scoring pretest Conspiracy Code students scoring below 60% correct and 60% or

higher correct on the pretests and posttests.

Figure 5 shows a decline in the below 60% correct group of 13% and an increase in the

60% or above correct group of 13% for the low scoring pretest group.


Low Scoring Pretest Conspiracy Code Students


21

Figure 6 shows a decline in the below 60% correct group of 13% and an increase in the

60% or above correct group of 13% for the higher scoring pretest group.


High Scoring Prettest Conspiracy Code Students

Other Analyses

In addition to the comparison of the low scoring and higher scoring group, statistical

analyses were also computed for the following groups:

• Males and Females

• Free/Reduced Lunch and No Free/Reduced Lunch

• English as a Second Language and Non-English as a Second Language

• Racial Groups

• Grade 9 Students and Grade 10 Students

None of the comparisons were statistically significant or were so slight as to not warrant

reporting. The results suggest that the Conspiracy Code: Intensive Reading program

has similar positive effects across all student sub-groups listed above.


22

Reading Attitude Survey Analysis: Conspiracy Code Students

An attitude survey was developed to assess students’ attitudes toward reading in

general, attitudes toward reading in the Conspiracy Code program, and attitudes about

personal reading progress. For each test item, students were given a statement to read

such as the following:

• The questions and discussions in Conspiracy Code cause me to want to

read more.

• I like to read about solving problems.

• I think my reading ability is improving.

Students were asked to respond to each of the statements by choosing one of the

following responses:

1. Strongly Agree

2. Agree

3. Unsure

4. Disagree

5. Strongly Disagree

There were a total of 36 statements on the survey. The 36 items were developed to

assess 3 different aspects of the students’ attitudes as follows:

• Does the student exhibit a positive attitude toward reading the Conspiracy Code materials?

12 Statements

• Does the student exhibit a positive attitude toward reading in general?

14 Statements

• Does the student believe his/her reading is improving? 8 Statements

The attitude survey was administered about half way through the semester (mid-March)

and again at the end of the program (mid-May) to the Conspiracy Code students. An

attempt was made to administer the survey to the Control Group students at both time

periods but confusion in the scheduling of the assessment resulted in a very small

number of surveys being completed by Control Group students. Thus, the following

analyses include only Conspiracy Code students.


23

The Conspiracy Code students took the survey via the internet. A beginning of the

program survey was not conducted since students would not yet have experienced the

Conspiracy Code program. In addition, starting students on a new program that was so

different from their typical classroom activities was not a good time to add an additional

activity. Thus, the survey was administered half-way through the program and again at

the end of the program. A total of 68 student surveys were matched from pretesting to

posttesting.


24

Table 8 compares the differences from mid-study to end-of-study. It is important to

note that lower scores on the attitude scale indicate agreement with the statements on

the survey. Thus lower scores indicate more positive attitudes.

A matched pair t-test was used to compare student scores on each of the three subscores

as well as the total test score. Students’ attitudes toward reading in the Conspiracy Code

program, attitudes about the improvement in reading ability, and the total survey scores

all improved statistically significantly (≤.0001) indicating a difference that would occur

by chance less than once out of 10,000 repetitions. While there was not a statistically

significant increase in attitudes about reading in general, the scores for this subarea did

improve from pretesting to posttesting. The effect sizes for the increase in attitudes

about personal reading improvement and the total survey score were medium. The

effect size for attitudes toward reading in the Conspiracy Code program was large.

Table 8

Paired Comparison t-test Results Mid-Course and End-of-Course Comparisons

Conspiracy Code Students* (N=68)

Attitude Toward:

When Given

Mean Score SD t-test Significance Effect Size

Mid-Year 3.10 .72 Conspiracy Code Reading End-Year 2.60 .68

6.379 ≤.0001 .71

Mid-Year 2.36 .46 General Reading End-Year 2.26 .51

1.719 Non-‐Sig. --

Mid-Year 2.47 .69 Reading Improving End-Year 2.17 .55

3.684 ≤.0001 .52

Mid-Year 2.64 .54 Total

End-Year 2.35 .53 4.675 ≤.0001 .54

*Lower scores indicate a more positive attitude.


25

Figure 7 provides a graphic view of the differences in mid-study and end-of-study

average scores for the Attitude Survey responses. All of the scores indicate a more

positive attitude at the end of the program when compared with mid-course results.

Figure 7 Comparisons of Attitude Responses at Mid-Study and End-of-Study

Conspiracy Code Students*

*Lower scores indicate a more positive attitude.


26

Think Along Assessment

The demands of the 21st century will require high levels of literacy for students to

succeed in school, in work, and in a world in which increasing and constant levels of

communication are made possible and expected through varied, new technologies. To

meet these demands, students must learn to be highly effective in understanding varied

texts and media.

An additional assessment was conducted at the end of the study to determine if students

in the Conspiracy Code Group and the Control Group differed in their ability to think

about their reading as they were reading. A technique has been developed in which

students are asked to periodically write their thoughts about what they are reading as

they read. The places to write are designated and the students are asked to write in

response to the prompt, “What are you thinking now.”

The Think Along Assessment for this study was composed of three passages whose

lengths were 207 words, 149 words, and 161 words. The first passage was a narrative

about a robbery; the second was an expository passage about the invention of the laser,

and the third was an essay about taking personal responsibility. The first passage

included three places at which students were asked to write their thoughts, and the

second and third passages each included 2 Think Along opportunities. Thus, there was a

total of 7 Think Along writing prompts.


27

Each response was scored on a 3 point scale as indicted below. Since there are 7 Think-

Along blanks, the total possible score will be 21 points.

0 No response or a minimal one or two

word exclamation Less than minimal

1 Response is unrelated to text and/or is primarily a regurgitation of what has been read

Minimal

2 Response is clearly related to text and includes thinking about the text

Satisfactory

3 Response is clearly related to text and indicates a focus on major points in the text. The thinking shows a connection between background knowledge and text

Excellent

The assessment was administered to 42 randomly selected students from the Conspiracy

Code Group and 52 randomly selected students from the Control Group. The scoring of

the Think Along Assessments was done by a reading curriculum specialist who had

several years of experience in developing and scoring these sorts of assessments. The

scoring was done by placing all of the papers in a single stack in no particular order. The

scorer did not, therefore, know whether a Conspiracy Code student or a Control Group

student was being scored. After the scoring was completed, the papers were sorted in

two groups, Conspiracy Code and Control Group.


28

A comparison of the scores is provided in Table 9. An independent sample t-test was

conducted to determine if there was any significant difference between the scores of the

two groups. The table shows that the Conspiracy Code students scored a higher average

score than the Control Group students and the difference was significant (≤.03)

indicating a difference that would occur by chance less than three times out of 100

repetitions.

Table 9 Comparison of Think Along Assessment Scores Conspiracy Code Group and Control Group

Tes

t

Tes

t F

orm

Nu

mbe

r of

Stu

den

ts

Sco

re

SD

t-te

st

Sig

nif

ica

nce

Eff

ect

Siz

e

Think Along Assessment

Conspiracy Code End-Year 42 68.9% 18.5%

Control Group End-Year 52 60.0% 20.2% 2.193 ≤.03 .46


29

Figure 8 provides a comparison of the percentage of Control Group and Conspiracy

Code students scoring below 60% correct and 60% or higher correct on the Think Along

Assessment. The figure shows that there is a 17% difference between the two groups

with more Conspiracy Code students scoring above 60% correct and 17% fewer

Conspiracy Code students scoring below 60% correct.

Figure 8 Comparisons of Think Along Assessments

Conspiracy Code and Control Group Students


30

Conclusions

Five research questions guided the study. Conclusions based on the results are provided

for each question.

Research Question 1

Is the Conspiracy Code: Intensive Reading program effective in

improving the reading comprehension and reading vocabulary skills of

struggling readers in grades 9 and 10; and, is it more effective than an

alternate program used by students in a control group?

The results show that the students in the Conspiracy Code program made statistically

significant reading score improvement from pretesting to posttesting. There was an

increase of 16 standards score points from 281 to 297. The percentage of students

scoring below 60% correct from pretesting to posttesting declined by 13% and the

percentage scoring 60% or higher correct increased by 13%.

The results for the Control Group students were disappointing in that it was expected

that the Control Group students would make at least significant gains over the course of

the semester. However, while the Control Group was statistically much higher on the

pretest comparisons than the Conspiracy Code Group, that advantage disappeared by

the end of the program. The higher scores for the Control Group students on the

pretests were expected once the information regarding the way the groups were

assigned to either group were revealed. However, the Control Group showed no

significant differences from pretesting to posttesting. In fact, the scores for the Control

Group actually declined slightly. This may seem very unusual until one examines Florida

Comprehensive Assessment Test in Reading (FCAT) test results for grade 9 and 10

students over a ten year period.

Reviewing the percent of grade 9 students scoring at level 3 or higher on the FCAT

Reading test each year from 2001 to 2010 indicates some increase of students scoring at

level 3 or higher each year. However, the average increase was only 2 percent per year.

At grade 10 there was an increase in only 3 years, a decrease in 4 years, and no change in


31

2 of the years. The average increase for grade 10 students over the period was less than

one percent (.03%). It is important in examining these results to recall that the FCAT

Reading test results include all students in the state who took the tests while the

Conspiracy Code and Control Group student test results represent a population of below

average students.

The gains made by the Conspiracy Code students are even more impressive when

compared to results on the FCAT Reading. While the FCAT test and the reading

comprehension test used in this study use different standard scales both scales use 300

as an average score. However, the FCAT scale covers 400 score points from 100 to 500

while the Reading Assessment scale used in this study covers a slightly smaller range

from 150 to 450. While the differences in score points are a bit different, a comparison

of gains on the two tests in standard score points and percent of improvement are

revealing.

The total sample of all grade 9 students in Florida, including students at all reading

ability levels, increased in FCAT standard score points from 312 in 2009 to 317 in 2010 ,

a gain of 5 standard score points. Using FCAT scores and the five point achievement

scale used to interpret FCAT reading scores, the results showed that from 2009 to 2010,

the percent of students in grade 9 in the entire state scoring in the top three

achievement levels declined from 54% to 48%.

The comparison with Conspiracy Code students, all of whom were in the program

because of poor reading ability, shows that their gains on the Conspiracy Code Reading

Assessment in standard score points from pretesting to posttesting was 16 points and

the percent of students scoring at the highest levels increased by 13% from pretesting to

posttesting. There are differences in the two tests used to make these comparisons and

there are differences in the populations included in each comparison, but the differences

are quite dramatic in emphasizing the significance of the gains students in the

Conspiracy Code program achieved.


32

The results provide very positive evidence for research question 1.

Conspiracy Code: Intensive Reading improved the reading achievement of

struggling readers. The results also reveal that the students in the

Conspiracy Code program achieved significantly better results than

students in a control group.

Research Question 2

Is the Conspiracy Code: Intensive Reading program equally effective

in improving the reading comprehension and reading vocabulary skills of

the lowest performing students to the same extent as higher performing

students?

The results showed that Conspiracy Code: Intensive Reading students who scored at the

lowest levels on the Reading Achievement test at pretesting made statistically significant

gains from pretesting to posttesting and the effect size for the total score gain was large.

The highest scoring students on the pretest group also increased their scores from

pretesting to posttesting; however, only the vocabulary scores reached statistical

significance.

The total pretest to posttest for the low scoring pretest group showed a statistically

significant gain in standard score points for the total of 23 standard score points. The

high scoring group achieved a gain of 8 standard score points on the total test score.

While the high scoring group gain was not statistically significant, the result is quite

positive when compared to the annual gains for grade 9 and 10 students on the FCAT

Reading test score results.

Perhaps of more interest are the results showing that the lowest scoring group decreased

the percent of students scoring below 60% correct from pretesting to posttesting by

13%; and, the percent scoring 60% or higher correct increased by 13%. The results also

show that the high scoring pretest students achieved the same 13% decrease at the below

60% correct and the increase of 13% at 60% and above correct.


33

The results provide very positive evidence for research question 2. The

Conspiracy Code low scoring pretest group made at least the same gains

and perhaps larger gains than the higher pretest scoring groups. Both

groups made gains but the low group’s pretest gains were statistically

significant while the high pretest group gains were not. However, the

decrease in lower scores and the increase in higher scores in both groups

were exactly the same.

Research Question 3


improving students’ attitudes toward reading?

This attitudinal assessment of students is difficult and is especially problematic with

high school students. The assessment survey used in this study consisted of 34 items.

The results showed that the student attitudes toward reading in Conspiracy Code

improved statistically significantly from the midcourse to end of course surveys and that

the effect size was large. The student attitudinal results toward reading in general

showed a small improvement but the change was not statistically significant. However,

when the test items asked about whether the students felt their reading was improving

there was a statistically significant difference and the effect size was medium.

The total score summing all three categories showed improved student attitudes at a

statistically significant level with a medium effect size.

The results provide positive evidence for research question 3. The

Conspiracy Code students had more positive and statistically significant

differences from the middle of the course to the end of the course in their

attitudes about reading in Conspiracy Code and in their attitudes about

improving as readers. Their attitudes about reading in general were more

positive but the improvement was not statistically significant.


34

Research Question 4


improving student’s reading/thinking strategies; and, is it more effective

than an alternate program used by students in a control group?

An attempt was made to assess each student’s use of effective reading strategies. A

Think Along Assessment was administered to a random sample of Conspiracy Code

students and Control Group students. The assessment was designed to assess how each

student used comprehension strategies by writing their thoughts at designated points

during reading activities. The results showed a statistically significant difference

between the two groups favoring the Conspiracy Code students. Seventeen percent more

Conspiracy Code students than Control Group students scored 60% or higher correctly

on the Think Along Assessment.

The results provide positive evidence for research question 4. The

Conspiracy Code students were more effective users of comprehension

strategies than were the Control Group students.

Research Question 5

Were any achievement gains made by Conspiracy Code: Intensive

Reading students influenced by demographic variables including gender,

grade level, ethnic background, socio-economic status, or first or second

language dominance?

There was interest in determining whether program effects would be different for males

than females since the program followed a game structure and there was some interest

in whether the use of the computer could result in greater gains for males. Only very

small gender differences were found and they favored female students. Grade level

differences were studied but there were so few grade 10 students in the Conspiracy Code

group that those results were not reliable and even though the group was small the

results did not favor grade 10 students over grade 9 students. Ethnic background,


35

socio-economic status, and first or second language background showed only very small

differences.

The results provide evidence that the Conspiracy Code: Intensive Reading

program was equally effective for all students.

Ccir efficacy studies

Software

Transcript of Ccir efficacy studies