Capstone Final

Language and Content Learning Using a Scaffolded Study Guide 1

Language and Content Learning Using a Scaffolded Study Guide

Anthony McConnell

Capstone Project

Western Governors University

In Partial Fulfillment of the Requirements for the Degree of

Master of Arts, English Language Learners

October 14, 2010


Abstract Current best practices dictate that English language learners benefit from a variety of

supplementary educational material. This research evaluates the effectiveness of one such

supplementary material, the scaffolded study guide. The study guide has been designed to help

students with both language and content objectives. The research was performed in an ELL

American History classroom with twenty-three students from variety of cultures speaking a wide

range of native languages. Data was gathered using field notes for observations, a student

questionnaire, the scaffolded study guide, a scoring rubric, and a post assessment. The data was

then analyzed using both qualitative and quantitative methods to find correlations between

student learning and performance on the study guide.


Table of Contents

CHAPTER 1: INTRODUCTION 7

Introduction 7

Approach and Setting 7

Selection of The Topic 8

Supplementary Educational Materials 10

CHAPTER 2: PROBLEM STATEMENT 12

A Demonstrated Need 12

Lack of English Ability 12

Lack of Content Knowledge 13

The Challenge 14

Meeting The Challenge 14

Research Questions 16

CHAPTER 3: LITERATURE REVIEW 17

Introduction 17

Content Based Instruction 18

CALLA 19

The Forsee Approach 19

SDAIE 20

Sheltered Instruction Observation Protocol 21

Using Supplementary Materials 22

Scaffolded Outline 22

Graphic Organizers 23


Defining Vocabulary and Cloze Sentences 23

Writing to Summarize and Learning Logs 24

Conclusion 25

CHAPTER 4: METHODOLOGY 26

Research Approach 26

Role and Bias 26

Sampling Technique 27

Participants 27

Obtaining Needed Permissions 28

Data Collection Methods 28

Participant Observation 29

Questionnaire 29

Student Performance 30

Instruments for Data Collection 30

Field Notes 31

Questionnaire 31

Scaffolded Study Guide 32

Rubric for the Scaffolded Study Guide 33

Post Assessment 33

ACCESS or W-APT Screener Scores 34

Reasoning for Methodology 34

Research Procedure 36

Qualitative Analysis Methodology 37


Classification and Categorizing 37

Quantitative Analysis methodology 38

Correlation 38

Triangulation 39

Integrity of the Data 39

Differences in Planning and Implementation 40

CHAPTER 6: RESULTS AND ANALYSIS 41

Outline of Results 41

Qualitative Results 41

Participant Observation 41

Questionnaire 42

Scaffolded Study Guide 43

Quantitative Analysis 44

Post Assessment and Study Guide 45

Post Assessment and Student Attitude 46

Study Guide and Student Attitude 47

Study Guide and Language Proficiency 48

Student Attitude and Language Proficiency 49

Post Assessment and Language Proficiency 50

Answering the Research Questions 51

CHAPTER 7: DISCUSSION OF RESULTS 53

Research Overview 53

Summary of Findings 53


Solutions to the Problem 53

Implications and Limitations 54

Strengths of the Research 54

Weaknesses of the Research 54

Problems with the Research 55

Factors That Skewed Findings 56

Improvements for Future Research 56

Critique of Masters Degree Experience 57

References 58

Appendix A-Data Collection Instrument – Field Notes Observation Protocol 61

Appendix B-Data Collection Instrument – Questionnaire 62

Appendix C-Data Collection Instrument – Scaffolded Study Guide 63

Appendix D-Data Collection Instrument – Post Assessment 69

Appendix E-Data Collection Instrument – Scoring Rubric for Study Guide 72

Appendix F-Data Collection Instrument – Qualitative Study Guide Questions 73

Appendix G-Informed Consent Form 74


Chapter 1

Introduction

English Language Learners face many challenges in American schools. In many cases

they are thrust into a culture that is completely foreign to them. This includes the culture that

surrounds education as well. It is expected that a student regardless of his or her English ability

be given comprehensible instruction in all subject areas. For secondary schools this can pose a

significant challenge. Many ELL students come to American high schools with little background

of subjects and content knowledge. Often times there may have been gaps in the student’s

education of several years. In social studies this is compounded by the fact that the material is

mainly centered on American history and culture and the content area is English dependant.

While it is good that these students receive this specialized instruction there are still

questions about how to accommodate the wide range of abilities within the class. The obvious

problem is that these students have varying levels of understanding of the English language.

Because of this it is necessary to provide ample supports to facilitate learning.

Approach and Setting

This was an action research study using a mixed method approach to data collection and

analysis to evaluate one such support. The purpose of the research is to find out if a scaffolded

study guide is a viable supplementary material for students. To accomplish this a study guide

was created based on a unit in American History. This was then given to the students at the start

of the unit. The students continued to use the study guide and complete various sections

throughout the unit. At the conclusion of the unit the students took a post assessment and

completed a questionnaire about their experiences using the study guide.


This research was done with an ELL American History class. The class is made up of 23

students from 8 different countries. There are ten students from Burma. This is the largest

subgroup represented in the class. These students speak 6 different native languages and many

have been in the United States for less than 2 years. Their English proficiency ranges from

beginning to intermediate. The location for the research was my classroom. This research took

place during regular school hours and the regular class meeting time.

Selection of The Topic

The topic for this study arose out of necessity. As a teacher of English language learners

I am constantly looking for new and inventive ways to engage students in the learning process.

Teaching sheltered English content classes presents the challenge of combining language

instruction and content instruction. In a sheltered content course language and content are not

taught separately. There are two different learning objectives, one for language and one for

content. However, the instruction for each one takes place simultaneously. This means that

materials used in the class must accomplish both functions.

This is a monumental challenge for teachers of English language learners. Prior to

teaching ELL social studies I taught regular education social studies courses. These included

world geography, American history, and government. The focus of every lesson was on content.

There were still some language objectives mixed in mostly related to improving writing ability.

Now switch to social studies content courses for English language learners. On a daily basis I

prepare for content instruction and language instruction for students who speak several different

native languages and who are at varying levels of English. The expectation is that these students

all accomplish the same level of learning as their English-speaking peers. They are held to the


same standards, take the same standardized tests, and are expected to achieve the same learning

objectives.

How is this possible? This is a question asked over and over among teachers,

administrators, and policy makers. The simplest answer is to make learning student centered.

When I first began teaching English language learners I was trying to help another content area

teacher with some problems she was having with ELL students in her class. The students were

failing the class and the teacher reported that they showed little effort or interest. To the teachers

credit she approached me about ways she could reach these students. To get an idea of where

she was at and what she had already tried I asked her what she does above and beyond the

regular classroom instruction to help the students understand the material. Her response was,

with a straight face, “I’ve tried talking really loud.” This teacher was essentially sitting the

students in the front and speaking louder thinking that this might in some way help these students

learn.

As bad as that sounds I don’t think it is very uncommon in regular education classes.

Teachers are faced with growing class sizes, as many as 35 in a class in some cases at the high

school I teach at. There are a variety of issues that they deal with in a 55-minute class ranging

from behavior problems to intercom interruptions. And finally, in many cases they have not had

the necessary professional development to educate them in instructional practices for English

language learners. This creates a lose-lose situation for teachers and more importantly for the

students.

This example demonstrates why content instruction for ELL students must be entirely

student centered. There is no room for teacher-centered instruction in an ELL setting. The needs

of the students are too diverse. A lecture format for a regular education American History course


may be justified at times. However, this type of format for English language learners can

accomplish nothing but frustration for both the students and the teacher.

Supplementary Educational Materials

Student centered teaching materials do not create themselves. This is why I am always

searching for and creating materials that I think can accomplish both language and content

objectives. This is why I chose this specific topic for this research study. Content instruction for

English language learners is directly related to the quality of the supplementary materials that are

available to the students. This means textbooks, assignments, and any other material used in the

classroom. A student cannot be expected to learn from a book that they don’t know how to read.

This is basic and not subject to any reasonable argument. Imagine someone was given a history

textbook in Russian and was asked to learn and then report back what they had learned about

Russian history. However, they don’t speak or read Russian. How successful can we expect

someone to be in that endeavor?

Now, is it possible that proper support and supplement can be supplied to help that person

learn Russian and Russian history simultaneously? I believe it is. But it is all about the types of

supports that will be offered. If they are good supports then success can be expected. If they are

poor supports then failure can be expected. Learning will take place only with excellent

supports.

It is for these same reasons that this research is important to the discipline of English

language teaching. Supports must be evaluated for their effectiveness. If a support is not

producing desired results then it must be modified or possibly eliminated from instruction. We

find out the value of a support by doing research such as this project. If this proves to be an

effective method then it may be able to be modified and implemented into other subjects and


possibly even for ELL students in regular education classrooms. On the other hand it may be

eliminated if no benefit to the students is shown. The problem is finding out what works.


Chapter 2

Problem Statement

A Demonstrated Need

The need for specialized student centered instruction for English language learners has

become more apparent in the last two decades. This has led to the development of a variety of

educational models such as SDAIE or Specially Designed Academic Instruction In English as

well as SIOP or Sheltered Instruction Observation Protocol. These educational models aim to

provide students with content knowledge as well as English instruction simultaneously.

There are many aspects to these models such as dual content and language objectives,

comprehensible input, and authentic assessments. All of these aspects intend to drive both

content and language learning. A vitally important aspect of any sheltered instruction program is

providing appropriate supplementary materials that enhance students’ learning (Echevarria,

Vogt, and Short, 2008). The needs for these programs particularly at the secondary level arise

from two very specific factors.

Lack of English Ability

This is the most obvious factor that plays into this problem. Many ELL students come to

the United States with little or no English ability. This creates a situation where they are

inherently limited in their access to our educational system in most instances. This problem is

compounded at the secondary level. An English language learner in an elementary or even

middle school setting has the benefit of time. They will have several years of instruction in

English and content that can prepare them for high school or college.

This is a luxury that many students who come to this country in their teens do not have.

These students enter high school with limited English ability. Many of them have also had gaps


their education. These students have only a few years to master English and content before they

are at graduation age. Research has shown that it takes 3-5 years to develop conversational

English and 5-7 years to develop academic English (Coelho, 2004). With that fact it becomes a

daunting task to make sure that English language learners are getting the education they need to

be successful in a four year high school.

Lack of Content Knowledge

Further complicating this is the need to keep students current on content knowledge and

graduation requirements while at the same time having them learn English. If secondary

students only focused on English Language acquisition they would undoubtedly fall behind their

English speaking peers in content subjects (Freeman and Freeman, 1988). This creates the

problem of teaching English and content simultaneously.

One of the things that regular education teachers count on is that students who have

grown up in the American school system have been at minimum exposed to important concepts.

For instance, in social studies, by the time a student reaches a junior year American history class

they are expected to know certain things. They may not be able to tell you everything George

Washington did. However, the vast majority can recognize his face and tell you he was the first

president. They may not be able to tell you very much about the Boston Tea Party or the Boston

Massacre but the mention of those things strikes a familiar tone with them. Even that basic

familiarity gives the teacher a jumping off point for instruction. On the other hand when I first

taught the Boston Tea Party to my ELL American History students I spent half a class period

explaining how it was different from a birthday party. Not one of the students in the class had

ever heard of the event before and the first association they make with the word party is a

celebration. The point is that when students come from another country we cannot expect them


to have any background whatsoever in the content we are teaching. This is particularly true in

social studies.

The Challenge

These two causes described above present a unique challenge. This is a challenge for

students, teachers, and administrators. These students are here and they have a right to a high

quality education. The problems are not small and the challenge is substantial. However, it is

the responsibility of the teacher and our educational system to bridge the gap and make sure that

students are receiving language and content instruction as rigorous as their English speaking

peers.

This challenge is met in a variety of ways. School districts establish special programs for

these students such as the one I teach in. Where a regular American history class may have 35

students I have 23. This is a way that the school is helping meet the challenge. The school is

creating an environment that allows for more individual student contact time with the teacher.

The challenge for the teacher is how to use that time and what instructional practices are

most effective. It is the teacher’s responsibility to combine language and content instruction.

This is accomplished by meshing best practices in content area education with best practices in

language teaching. In this particular study one of the possible instructional methods that

incorporate these is being evaluated. This is the scaffolded study guide.

Meeting the Challenge

The scaffolded study guide will consist of defining vocabulary, a graphic organizer, cloze

passages for vocabulary enforcement, a scaffolded outline, and writing prompts for student

reflection. This will all be combined into one packet and given to the students at the beginning


of a unit on the American Revolution. Although the sections of the study guide will match daily

lessons students can work at their own pace to complete it.

The intention of this study guide is to allow students access to content knowledge while

at the same time developing English ability. In each activity of the study guide both of these

objectives are taking place. The reason that I call this a scaffolded study guide is that I have also

tried to include learning strategies such as note taking. This study guide allows the students to

practice various learning strategies within the content and language objectives. The purpose of a

scaffold is that it is eventually removed. It is a temporary support until the learner can

accomplish the goals independently. If the study guide proves to be effective future versions for

different units may be expected to have less supports and more independent student practice.

The first step to evaluating the effectiveness of this study guide was to ask some key questions

that would guide the research.


Research Questions

1. What are ways that we can scaffold language learning and teach content at the same time

using supplementary educational materials?

2. Furthermore, how can we incorporate vital learning strategies into those materials and

activities that will help students become more efficient and self aware in their own

learning?

3. Does a scaffolded study guide provided to the students for use throughout a unit provide

support for content and language while at the same time teaching learning strategies?


Chapter 3

Literature Review

Introduction

There are two main approaches to language learning. There is the synthetic approach, a

bottom up method in which language is constructed through grammatical, lexical, phonological,

and functional units of language. The analytic approach on the other hand is a top down system

where appropriate materials are selected and the learner extracts rules of language through

meaningful experiences (Fromkin, Hyams, & Rodman, 2007). One of the more current teaching

perspectives focuses on communication in the target language and is know as Communicative

Language Teaching or CLT. This approach suggests that meaning and communication be the

main goals of language instruction and as such meaningful opportunities for authentic language

use be provided (Celece-Murcia, 2001).

Research shows the important role that English oral development plays in the overall

process of English language acquisition. With better proficiency English Language Learners are

better able to engage in other areas of academic learning (Genesee, Lindhom-Leary, Saunders, &

Christian, 2005). Oral language proficiency used for communication and day-to-day needs is

known as BICS or Basic Interpersonal Communication Skills. This type of language usually

develops in 2 to 3 years. On the other hand CALP or Cognitive Academic Language Proficiency

takes between 5 and 7 years for development (Coelho, 2004).

This presents challenges for teachers and school districts as the number of English

Language Learners continues to rise in the United States. Statistics show that the number of

English language learners has doubled in the last decade. Continued Growth is projected with

ELL students making up 40% of the K-12 student population by 2030 (Washburn, 2008).


Unfortunately, teacher education programs have not kept up with the growing population of ELL

students. It is likely that by 2050 there will be an ELL student in every teacher’s classroom in

the United States. However, an analysis of the most widely used teacher education textbooks

revealed that on average, the texts only devote 0-2 percent of their content to discussing these

students (Watson, Miller, Driver, Rutledge, and McAlister, 2005).

There have been several attempts to correct this deficiency. The Training all Teachers

project was designed to infuse ELL issues throughout core curriculum and train teachers and

other service personnel (Meskill, 2005). Training mainstream teachers seems to be an ongoing

issue with many sources and studies dedicated to it. For instance, a survey of 327 teachers in the

Midwestern United States found strong support for the need for specialized instruction for ELL

students. However, the support was not as apparent for practical application of the methods

(Karathanos, 2009). The survey suggests a clear link between university ESL preparation and

increased support of theory and practice of specialized instruction.

Content instruction is particularly prevalent in secondary classrooms. Improving

academic performance for ELL students requires implementing high quality, sheltered

instruction steered by research (Echevarria, & Short, 2005). There are a variety of current best

practice models, which describe instruction for English Language Learners. These best practices

involve a number of strategies and techniques that make content comprehensible and provide

opportunities for practice using academic English (Echevarria, Vogt, and Short, 2008).

Content Based Instruction

In content-based instruction the sequence of language learning is dictated by content

material. There are a variety of models such as thematic instruction, sheltered instruction, and

adjunct instruction (Brinton, Snow, and Wesche, 2003). Thematic is a primarily a language


curriculum that is centered on a content topic. Sheltered instruction uses modified content to

make input comprehensible for language learners. Adjunct instruction requires two separate

courses. One of which focuses on content and one that focuses on language (Crandall, 1994).

Since the setting for this research project is a sheltered instruction social studies classroom that

will be the main focus of the various approaches.

CALLA

CALLA is an instructional model, which has applications regardless of which content

approach is used. CALLA stands for Cognitive Academic Language Learning Approach. This

idea is based heavily on Cummins theories of BICS and CALP and emphasizes the learning

process as opposed to teaching strategies (Chamot and O’Malley, 1994). Learning strategies

should be the foundation of any language-learning lesson. This includes metacognitive strategies

such as self-evaluation and organization. CALLA is not intended to substitute for mainstream

content instruction as is the intention of sheltered instruction programs (Chamot and O’Malley,

1987). Although CALLA is it’s own separate instructional model it’s concepts can be used for a

sheltered instruction classroom since the primary goal is to bridge the gap between

conversational and academic language. Sheltered instruction students can be taught specific

learning strategies for monitoring their own progress. Research has shown that students who

continually monitor their own learning are better able to adapt when learning becomes more

difficult (Crandall, Jaramillo, Olsen, and Peyton, 2002).

The Foresee Approach

The Foresee Approach gets its name from four words that start with the letter “C” in the

concept of “Communication, Cognitive Academic Language Processing, and Content

Instruction in the Classroom.” This approach to content instruction is an extension of CALLA


(Kidd and Marquardson, 1994). The Foresee Approach has three primary aspects to instruction.

It includes a learning strategies component, a content component, and a language component.

This approach to content ESL emphasizes not only language but also metacognitive strategies

that learners can use across subjects and disciplines (Kidd and Marquardson, 1994). The

architects of this approach use the methods of CALLA to inform content instruction so that the

learners are not only focused on the “what” of learning but also the “how” (Kidd and

Marquardson, 1994).

SDAIE

Sheltered instruction is sometimes referred to as Specifically Designed Academic

Instruction in English or SDAIE. SDAIE is grade level subject matter instruction in English

specifically designed for speakers of other languages. It is not watered down curriculum but

rigorous instruction provided at an appropriate grade level (Sobul, 1995). SDAIE offers an

excellent opportunity for monolingual English speaking teachers to acquire the skills and

strategies to impart effective instruction (Cline, & Necochea, 2003). SDAIE is respectful of the

learner and provides a safe and nurturing environment for learning. It also provides primary

language supports and offers students multiple access points to content units and lessons. It is

cooperative and interactive learning based on using a variety of visuals, materials, and

technology (Cline, & Necochea, 2003).

The ultimate goal of sheltered instruction is to transition students to the mainstream

classroom environment. It is important though that students be able to keep pace in content areas

with their English-speaking peers. If teachers were focused solely on language acquisition

students may fall so far behind they cannot catch up ( Freeman and Freeman, 1988). This is

particularly true in a secondary environment where content classes often build upon one another.


Traditional comprehension problems that may present themselves in regular classroom may be

alleviated in various ways. These include slowing down speech, placing an emphasis on

vocabulary, providing adequate wait time, and modifying instruction to the learner’s English

ability (Hansen-Thomas, 2008). Specific strategies combined with best instructional practices

provide the foundation for SDAIE and sheltered instruction.

Sheltered Instruction Observation Protocol

Sheltered Instruction Observation Protocol also known as SIOP is a research based

instructional model for sheltered English instruction and sheltered content instruction. It was

developed over 5 years of collaboration with researchers and teachers. It provides concrete

examples of features of sheltered instruction that enhance and expand instruction (Abadiano, &

Turner, 2002). In effective SIOP lessons there is a high level of student engagement and

interaction with the teacher, with other students, and with the text, which leads to elaborated

discourse and critical thinking (Echevarria, Vogt, and Short, 2008).

There are two distinct aspects of SIOP. One is the SIOP model which is the lesson

planning and delivery system. The other is the SIOP protocol, which refers to specific

instruments used to observe, rate, and provide feedback on lessons (Echevarria, Vogt, and Short,

2008). The SIOP model has been designed for flexibility and tested in a wide range of classroom

situations. It is not designed as a strict rigid program but as a framework for instruction. It

allows for teachers to keep favored techniques as well as add new elements to lessons and

instruction (Echevarria, Vogt, and Short, 2008).

Some features of SIOP instruction are making sure content is comprehensible, creating

language and content objectives for lessons, and providing frequent opportunities to practice and

demonstrate knowledge (Echevarria, Vogt, and Short, 2008). SIOP can be adapted to a variety


of content areas including social studies. The model provides direct support and access to

academic vocabulary through teaching content words, process words, and teaching English

language structure (Echevarria, Vogt, and Short, 2011).

Using Supplementary Materials

A common thread through all of the various approaches is using appropriate

supplementary materials for instruction. For ELL student’s supplementary materials should

provide access and entry to content material. By scaffolding materials you enable the learners to

perform at a higher level than they would be able to without support (Coelho, 2004). Scaffolding

allows for temporary supports that permit learners to do complex tasks before they would

otherwise be ready (Celece-Murcia, 2001). Lev Vygotsky first introduced the idea of the Zone

of Proximal Development. This is the place where students can do more than they would be able

to independently (Lightbrown and Spada, 2006).

Scaffolded Outline

It is important to teach English language learners to take good notes. In a study ESL

students who were taught to take notes outperformed their peers. This emphasizes the need for

explicit instruction and support in helping students learn to take good notes (Tsai-Fu and Wu,

2010) Note taking enhances reception by promoting learners to attend better to what they are

hearing and seeing (Borich, 2007). Teacher created outlines enable a student to become familiar

with a form of note taking while also providing support for texts and content material. Major

topics of the outline may be filled in already to provide guidance to the students. The students

then add other information as they read. This helps students to process information as well as

practice skills simultaneously (Echevarria, Vogt, and Short, 2008).


Graphic Organizer

Graphic organizers are visual representations of information. By using graphic

organizers students are able to see organization, structure, and relationships between information

and concepts (Celece-Murcia, 2001). Graphic organizers can take many forms such as a concept

map, cause and effect, timeline, or a compare and contrast. The use of a graphic organizer is a

key visual representation of ideas. Graphic organizers are also highly flexible as they may be

adapted for any content or lesson (Coelho, 2004).

Utilizing graphic organizers allows students with visual and spatial learning styles to

have access to complex texts and materials. The visual representation of information allows

ELL students to have a cognitive warehouse that facilitates recall and retrieval (Peregoy and

Boyle, 2001). Graphic organizers promote students literacy progression by developing

background knowledge and providing motivation for reading (Peregoy and Boyle, 2001). When

used prior to reading graphic organizers can be used as a reading guide. When used concurrently

with reading they can be a form of note taking which helps students understand text structure.

When used after reading they can be used as reflective tools to assess understanding (Echevarria,

Vogt, and Short, 2008).

Defining Vocabulary and Cloze Sentences

There is little benefit to selecting isolated vocabulary terms and asking ELL students to

copy them and define them. It is very likely they may not understand the words in the

definitions themselves, which then make the entire approach meaningless (Echevarria, Vogt, and

Short, 2008). In content classes many new vocabulary words are likely to be subject specific and

therefore require context for better understanding (Coelho 2004). Practicing learning vocabulary

through using context also facilitates independent learning. Vocabulary instruction that requires


students to predict meaning based on context clues is particularly effective for teaching low

frequency subject specific words (Celece-Murcia, 2001).

One method for providing vocabulary instruction in a meaningful way is through the use

of cloze sentences. Students are given sentences with strong contextual support for the

vocabulary word that has been omitted (Echevarria, Vogt, and Short, 2008). When using a

rational/purposive deletion technique content area vocabulary is omitted from the sentence

(O'Malley, & Pierce, 1996). Students may be provided a list of possible vocabulary options

or may brainstorm possible words.

Writing to summarize and learning logs

English Language Learners should be encouraged to write in English early and should be

provided opportunities to express their ideas in writing even in the beginning stages of English

development (Echevarria, Vogt, and Short, 2011). Providing scaffolding tools such as graphic

organizers or sentence fames for organizing key points will help ELL students write in the

content area (Echevarria, Vogt, and Short, 2008).

One important types of writing students use is writing to summarize. Students can be

provided opportunities to summarize important information presented in a text or graphic

organizer. This type of writing helps students look for important details and points when they

are constructing written material (O'Malley, & Pierce, 1996). A second type of writing is

the use of a learning log. This can be used as a tool of self-assessment. Students may be asked

to write and reflect on what they learned in a particular lesson, what worked best for them, what

was easy to understand, what was difficult and so on and so forth (O'Malley, & Pierce,

1996). Learning logs help develop metacognitive skills by encouraging students to express

their thoughts in writing and reflect on and make plans for their own learning (Coelho, 2004).


Conclusion

There are several approaches to content instruction for language learners. Many classes

demonstrate various aspects of a variety of approaches. For instance, there are several elements

of CALLA in the SIOP protocol. There are also several different strategies and objectives for

supplementary learning materials. These materials can be adapted to the specific needs of

English Language Learners. By combining research based instructional approaches with

research based learning strategies ELL students can have a higher probability of success in

content courses.


Chapter 4

Methodology

Research Approach

To answer the research questions I used a mixed method action research approach.

Action research in this situation is systematically gathering information about the way students

learn (Gay, Mills, & Airasian, 2009). Action research is appropriate here because it looks at an

authentic classroom situation from a holistic perspective to gain insight into one aspect of

instruction. This research approach is relevant to the problem and is the best method to answer

the questions posed by it. The mixed methods used for this research provide both quantitative

and qualitative data.

Role and Bias

I play a dual role in this research study as the researcher and the teacher. This benefits

this study because of my knowledge of the subjects. This is particularly true of using

observation as a technique for data collection. As the classroom teacher I want students to be

successful. It is my intention to provide them the best materials and opportunities to do that.

Therefore, I approached the data objectively to gain an accurate understanding of the success and

limitations of the study guide in question. I then use this to inform my teaching.

For the students there are a variety of biases present. All of the instruments used to

collect data are in English. The mere fact that they are learning English is a bias that I must

account for when analyzing the data. Also, there are various cultures represented within this

class. These cultures have varying views of school and the roles of students and teachers. When

making observations and collecting data this must be taken into account. Many of the biases

were minimized though. The students and I have had ample time so far this school year to get to


know each other. I have tried to foster a safe and nurturing classroom environment. There is

evidence that this has been successful by the students’ comfort level reading and participating in

class. This is true across all language groups and cultures. I continued to provide the students

with the safe environment they have become accustomed.

Sampling Techniques

The sampling technique used in this study was purposive sampling. The specific class

selected was based on the researchers knowledge of the class make up. Possible classes for

selection were ELL Federal Government, ELL English and ELL American History. The criteria

for selection were an appropriate sample size, the majority of students at the same grade level,

and a variety of different native languages spoken and cultures represented. For this study the

ELL American History class was selected for the following reasons. The class size of 23 is the

closest to the desired 20-student sample, which would be ideal for the study. The class is a

required junior year course. The majority of the students in the class are all at the same grade

level. This class has a variety of cultures and languages represented which will allow me to

analyze the data in terms of culture and language.

Participants

The participants share some common features yet also some drastic differences. The

class is composed of one 12th grade student, two 10th grade students, and twenty 11th grade

students. There are a total of 23 students in the class and the majority of them are juniors. Their

ages are between 16 and 19 years old. There are 8 males and 15 females in the class.

The students are from Burundi, Rwanda, Burma, Liberia, Nepal, The Democratic

Republic of the Congo, and Tanzania. The largest concentrations of students are from Burma.

There are a total of 10 students from Burma. The students from Burma also represent the largest


language segment in the group, which is Karen. Other native languages in the group are Kirundi,

Kru, Nepalese, Kinya-rwanda, and Swahili.

The vast majority of the students have been in the United States less than two years. No

student has been in the United States longer than three years. The students are primarily from

low-income families. One common thread among the students is that many of them have been

relocated here through various refugee organizations. These organizations relocate families and

individuals with refugee status from war torn countries to different places throughout the world.

In some cases they have had previous English instruction and schooling. In other cases such as

one particular student from Rwanda there has been gaps in schooling as long as 7 years.

Obtaining Needed Permissions

The research project was clearly explained to the students prior to being implanted. The

informed consent form for this project was also used. Students were given the form and returned

it. Obtaining permissions and consent in this research study was not difficult. It is simply good

teaching to look at your methods and student performance to help you inform instruction. In fact

it is required of me by my employers to do exactly what this research study intends which is to

evaluate my teaching methods. This is done on a daily basis with no risks to the students. This

project was no different. There were never any risks for the students associated with this

research.

Data Collection Methods

This is an action research project that utilizes a mixed approach to gather both qualitative

and quantitative data. Qualitative data is descriptive data that is usually narrative and visual

rather than numerical (Gay, Mills, & Airasian, 2009). Collecting qualitative data requires a fair

amount of fieldwork. In this case I was immersed in the research environment since I am the


teacher. Quantitative data on the other hand can be measured. For instance, student performance

on a task or an assessment. This research project utilizes several methods that will yield both

qualitative and quantitative data.

Participant Observation

When using observation as a data collection technique a researcher can be involved in the

setting or not involved at all (Gay, Mills, & Airasian, 2009). In this case I am teaching the class

so I was definitely a participant in the environment. This particular approach has the advantage

of allowing the researcher to develop relationships with the participants, which can provide

valuable insights. One drawback to this method is the possibility to become emotionally

involved and lose objectivity (Gay, Mills, & Airasian, 2009). This is something that I remained

conscious of as a researcher. These are my students and I am involved and invested in their

success. The best way to ensure that was to objectively view my work and this research to find

better ways to serve my students.

Questionnaire

A questionnaire is a written collection of self-report questions to be answered by

participants. This approach let me know how the students felt about a particular aspect of the

research, the study guide. The questionnaire allows the researcher to collect a large amount of

data in a short period of time (Gay, Mills, & Airasian, 2009). This type of approach must be

crafted carefully. The questions must be pertinent to the research and structured to eliminate

confusion and biases. The questions must be clearly written and carefully checked for any

errors.


Student Performance

The third approach that was employed looked at student performance. Specifically the

instruments that were used are the scaffolded study guide, a scoring rubric for the study guide,

and a post assessment. Using these items gives valuable insight into whether or not the students

were able to master the learning objectives of the unit. This approach can also provide data that

can then be analyzed along with my field observations and student responses to the

questionnaire.

Instruments for Data Collection

Field Notes

To record my observations I kept daily field notes. Field notes contain two types of

written information. Descriptive information is that which has directly been seen. Reflective

information is comprised of thoughts and reflections on observation data (Gay, Mills, &

Airasian, 2009). I looked at how the students were using the scaffolded study guide. I wanted

to see if it was easy for them to follow, or confusing. I also wanted to get an idea of how they

are using the information. For instance, were they using the study guide to enhance background

knowledge for the upcoming lesson? I recorded all observations to make sure that I was getting

the most complete picture of how the students were using the study guide.

To ensure that I recorded all pertinent data I used an observation protocol to guide my

fieldwork. The observation protocol consisted of the following topics.

Descriptive Observations

1. What will be my role during the observation?

2. What will the students be doing?

3. What interactions if any are happening between the students?


4. What are the distractions to the learning environment?

5. How are the students using the scaffolded study guide?

6. What challenges are the students having with the scaffolded study guide?

7. How did the class end?

Reflective Observations

1. Questions arising from the observation?

2. Insights and thoughts from the observation?

By using this observation protocol to guide my fieldwork I was able to compile

uniform written data of the class environment and the students.

Questionnaire

The questionnaire for this research project is designed to gauge student’s opinions of the

effectiveness of the scaffolded study guide. I wanted the students to evaluate their own

performance and decide if the study guide was something that was helpful to them during the

unit or not. Of all the data I could collect on this subject none may be more important than

student attitudes and feelings toward the item being researched. Current educational research

suggests that when students “buy into” an assignment or a lesson they perform at a higher level.

The questionnaire consists of five positive statements about the study guide. There is also a

section for students to add comments of their own. The statements for the questionnaire are as

follows.

1. This study guide was easy to use.

2. The purpose of the study guide was clear to me.

3. The study guide made it easier to understand the material in the unit.

4. The study guide was a useful tool to have when studying for the quiz.


5. I would like to use a study guide like this for all the units.

The questionnaire used a Likert Scale to measure the student responses. The student

chose which statement best applies to the question.

The choices are:

Strongly Agree (SA)

Agree (A)

Undecided (U)

Disagree (D)

Strongly Disagree (SD)

The student’s responses are converted to a number from 5 to 1 with 5 representing

(Strongly Agree) and 1 representing (Strongly Disagree). This questionnaire allows me

to understand if students have accepted the study guide as something beneficial to them.

The scores for each question were tallied and produced a raw score that was then

converted to a percentage.

Scaffolded Study Guide

The scaffolded study guide consists of a graphic organizer, cloze passages for

vocabulary, a scaffolded outline, and writing prompts for student reflection. When analyzing the

students’ performance on this item I asked four questions:

1. What sections gave students the most trouble?

2. Do student reflections indicate learning?

3. Were there sections where completion is inconsistent with directions?

4. Are there any additional things the student has added to the study guide?


These questions and assessments of the study guide are primarily to gain qualitative data on

how effective the study guide was put together. If there are flaws such as unclear directions or

tasks that are too complex then that will skew all the rest of the data that I collected. If it is

determined there were aspects of the study guide that were poorly constructed that must be taken

into account when analyzing all the other data.

Rubric for the Scaffolded Study Guide

The scaffolded study guide was scored using a rubric. The main sections of the rubric are

completion, accuracy, mechanics of grammar and writing, and organization of the writing.

These different categories were scored as excellent, good, approaching, or does not meet. After

scoring the categories for each student the scores were then converted to raw numbers with 4

representing (excellent) and 1 representing (does not meet). The totals for these sections were

added to give a raw score. This score was then converted to a percentage.

Post Assessment

Students were administered a ten question post assessment that was meant to measure

student mastery of content objectives. The questions are multiple choice with four possible

answers for each question. The questions on the post assessment were aligned to the text,

lessons, and the study guide. The post assessment quiz was scored with each student response

evaluated as correct or incorrect. Scores were calculated on the number of correct answers out of

ten. This produced a raw score that was then converted to a percentage. This instrument allows

me to see if the students have mastered the content objectives set out in the lessons of the

chapter.


ACCESS or W-APT Screener Scores

Since this is a research study that centers on English language learners it is only logical to

include students English proficiency level in the data analysis. The state is part of the WIDA or

World Class Instructional Design consortium. Students who are new to our school are

administered the W-APT placement screener. This screener gives a score for the student’s

English level. Every January all ELL students are administered the ACCESS test. This test is an

annual evaluation of English proficiency. All of the students in this class have a score either

from the W-APT screener or from the ACCESS test. Although the WIDA levels go from 1-6 the

state considers a student with a score of 4.8 to be English proficient. For calculation reasons I

made the score of 4.8 equivalent to 100% and converted all of the students test scores to a

percentage. This allows the language proficiency to be evaluated along side the other data.

Reasoning for Methodology

The methods and instruments chosen for data collection have been carefully selected to

give me a holistic view of the effectiveness of the study guide. The observations allowed me to

collect day-to-day qualitative data about students’ reactions and performance. The data was

collected using daily written notes of my observation experience. The written notes follow the

observation protocol outlined earlier. There were many types of specific data observed. First

was how much time the students spend on task with the scaffolded study guide. Second, how

they understand the directions and navigated the material. Third, how they worked with each

other using the study guide. Fourth, if the students were using the scaffolded study guide to

anticipate the next lesson.

The questionnaire allowed me to gather both qualitative and quantitative data. The

student’s responses to the questions were converted to numerical data. Each question was then


broken down individually and scores assigned to each question. A high numerical response to

the question indicates the student agrees with the statement. A low numerical response means

they do not. Using this data gives me an impression of whether or not the students felt the study

guide was an effective tool for their learning. This data is of particular importance because these

are the students that have been using it. They know whether or not it has been beneficial to

them. This data allows me to compare student attitudes of the scaffolded study guide to

performance. The total scores for each question were combined and then converted to a

percentage.

The data that I got from the scaffolded study guide was two fold. First, I got qualitative

data by analyzing the study guide for design flaws using the questions cited earlier. This data is

in the form of written notes made as I reviewed the study guides. This data reveals any possible

problems with the instrument itself.

Using the rubric allows me to evaluate the students’ performance on the study guide on a

consistent level. The scores of each section were then added together for a total score. For

instance, a raw score of 16 represents an excellent on the study guide. A score of 4 would

represent a performance that does not meet the expectation. Converting the scores of each

section to a total number gave me holistic data on the study guide. The total scores and

performance indicators break down as follows: 4 does not meet, 5-8 approaches, 9-12 good, 13-

16 excellent. When converting these scores to a percentage a raw score of 16 would convert to a

100% on the study guide.

Since the lowest score on any individual section of the rubric is a 1 the lowest possible

total score is a 4. Converting the results of the rubric to a numerical representation provided


quantitative data that I used to compare students and their performance on the study guide, the

questionnaire, and the post assessment.

The post assessment provided me with outcome data that was numerical as well. This

quantitative data let me see what the students learned from the unit. The students were assigned

a raw score based on number of items correct. For instance, a student with 8 correct items would

have a raw score of 8. The percentage score would then be 80%.

All of the raw numerical data from the questionnaire, the study guide and the post

assessment were converted to a percentage score for analysis. It is difficult to compare the raw

data numbers to each other. However, scaling all scores up to a percentage allowed for all

aspects to be compared with each other.

Combining the data qualitative data from the observations, questionnaire, and the study

guide with the quantitative data from the questionnaire, the scoring rubric, and the post

assessment allowed me to make an assessment to the effectiveness of the study guide in a

curriculum and answer the research questions.

Research Procedure

The following is the procedure used to conduct this research.

1. Review the Literature

2. Identify the best research approach.

3. Identify the best methods of data collection.

4. Create a plan to analyze the data.

5. Plan curriculum units.

6. Create the instrument to be studied (study guide).

7. Introduce the study guide at the beginning of a new unit.


8. Make field observations throughout the unit.

9. At the end of the unit administer the post assessment.

10. Administer the questionnaire.

11. Evaluate the scaffolded study guide for design flaws.

11. Evaluate the scaffolded study guide based on the rubric created.

12. Triangulate the data from observations, questionnaire, and student performance.

13. Make conclusions.

14. Report findings.

Qualitative Analysis Methodology

Classification and Categorizing

To analyze the field notes that I took during my observations I used a classification

technique. It is important to note that analysis of observations took place throughout the research

process though. I analyzed my field notes on a daily basis to gain insight and information that

informed my teaching as the unit progressed. Categorizing the data took place after all of the

observations have been done.

In a way the analysis process began before the research even started by designing the

observation protocol. By organizing the notes and information into categories or themes that

were informed by the observation protocol, connections appear more easily. For instance,

question 1 of the observation protocol looked at my role during the observation. I compiled all

of the notes taken during the research under this question and used it to form a post research

category. This was then repeated throughout all of the questions used in the observation

protocol.


This same method was employed when analyzing the scaffolded study guide. I created a

set of questions I used to qualitatively evaluate student performance on the study guide. These

questions became my post research categories for the data gathered from this instrument. After

analyzing the study guides of all the students and making notes I then took the information from

each student and grouped it together. This allowed me to see themes in the data.

The data from the questions on the questionnaire were converted to numerical data and

analyzed quantitatively. However, there is a part of the questionnaire that allows the students to

express their opinions of the study guide in their own words. This data was analyzed

qualitatively. I used the same method of analysis as with the other qualitative data. I looked for

themes within their free responses. I organized their responses into categories of positive and

negative. After the quantitative analysis of the questionnaire was done I was able to make

connections between the students free responses and their responses to the structured questions.

Quantitative Analysis Methodology

Correlation

The data collected from the questionnaire, the post assessment, and the scoring rubric for

the study guide were analyzed using a quantitative method. The main question that the research

should answer is whether the scaffolded study guide has a positive effect, negative effect, or no

effect on student learning and performance. As a researcher I looked for a correlation between

these variables. Data was gathered by analyzing student performance on the study guide and the

post assessment. The questionnaire also measured student attitudes toward the study guide.

To see if there is a correlation between the study guide and student performance on the

post assessment I used the product moment correlation coefficient technique. This technique is

also known as the Pearson r. The Pearson r is more useful for sample sizes less than 30, which


makes it an ideal choice for my research (Gay, Mills, & Airasian, 2009). The data gained from

student assessments is often continuous or interval data which also makes this technique an

appropriate choice.

To complete this analysis I used Microsoft Excel to input the data from the questionnaire,

the scoring rubric, and the post assessment. I analyzed these variables two at a time by selecting

an open cell and typing =correl and then selected the fields I wanted to analyze. After the fields

were selected I was given a correlation coefficient. A coefficient of 1 means there is a perfect

correlation and anything below gives me an indication of how closely the variables are related. I

repeated this procedure for all sets of data including the student’s language proficiency.

After this analysis has been done I used Microsoft Excel to create several different graphs

of the data. These graphs make it easy to see if there is a correlation between the variables for

each individual student.

Triangulation

The final method I used is to triangulate the analysis. This is where the qualitative can be

combined with the quantitative. I took all the quantitative analysis that I did and tried to see if it

matched what I had observed in the classroom.

Integrity of the Data

The integrity of the data obtained was protected at all times. No student names or other

identifying information was used. Also, all of the data collection instruments were in my

possession at all times after the research was carried out. Data from the instruments was put into

Microsoft Excel and all data is reported accurately and all calculations have been checked

several times to ensure validity.


Differences in Planning and Implementation

There were no differences between the methods or instruments that I intended to use and

what was actually used in the research. All of the data analysis was done exactly like I had

planned out. The differences came because of circumstances that could not be controlled or

foreseen. The initial time frame that I planned to implement this project was two weeks.

However, the project ended up taking several more days beyond that. Reasons for this were

challenges students were having, a fire drill, and an assembly. These things take time away from

class work and in turn the research.

I would describe it as real life meeting good planning. I had the entire two weeks

mapped out for what we would be doing and what would be happening in the class. Again this

does not always work out as planned. It was my intention to do all of the observation protocols

as the class sessions were taking place. However, distractions such as a student who was upset,

or a student who had no money for lunch required me to perform other duties of my job while

the research was taking place.

These factors forced an adaptation on my part to the changing circumstances of the day.

The image of me with my observation protocol on a clipboard was quickly changed to the reality

of me jotting notes on my hand and transcribing them later to the protocol form. Other than

these factors outside of my control there were no major changes in implementation of this

research.


Chapter 6

Results and Analysis

Outline of Results

The results of the data analysis will be presented in two parts. First the qualitative data

results and findings will be presented. This will include the analysis of the observations as well

as the study guide and questionnaire. Next the quantitative results will be presented and

analyzed. To do this I have created graphs to visually show the correlations between the data

sets. Finally, the data will be used to answer the research questions posed at the beginning of

this project.

Qualitative Results

Participant Observation

There are several interesting and positive results from the analysis of the observation

protocol. A few things when analyzed did not change such as my role during the observation. I

was always serving in the capacity of the teacher and was assisting students as needed. The

students were impressive with their time spent on task. When the students were using the study

guide they were very focused and engaged. The students assisted each other with various aspects

of the study guide as needed. I was glad to see the students spontaneously begin to help one

another on the different aspects.

Along with the positive there were also negative observations made as well. First, the

students had a difficult time in the beginning understanding that this was something they had to

be responsible for. For the first two or three days I kept extra copies of the study guide available

for the students anticipating this would be a challenge. Everyday for the first three days a couple

of students had forgotten their study guides or lost them. However, after reminding the class that


this was their work for the unit the the problem resolved itself. From the 4th day till the end of

the project I did not hand out one extra study guide. All students were prepared with their study

guide everyday. This is an encouraging aspect because it means students learned responsibility

and forced themselves to be organized.

A second negative observation was that much time was spent giving students directions.

I would explain the directions to the students and several of them would need individual help to

understand the expectations. This cut into instructional time and time that they could spend

working on the study guide. This is related to difficulty the students had in the writing aspects of

the study guide. For these students learning English writing is by far the most challenging aspect

of the language. The students spent a lot of time and had a significant amount of observable

difficulty with the writing tasks.

Questionnaire

The qualitative analysis of the questionnaire is based on the student’s opportunity to

respond in their own words about their feelings toward the study guide. This analysis is

inconclusive. This is due to the fact that only 3 students out of twenty-three used the opportunity

to offer feedback. This is not completely surprising given that writing is a difficult task for these

students. The comments from the students were as follows:

“Study Guide is helpful to study for the test.”

“I like the study guide”

“The hard thing was when you had to find your own words for the definitions. That was only

hard thing for me.”

Since only three students responded it is difficult to use this to either support or negate

the rest of the data gathered. However, for the student who said it helped study for the test there


was a noticeable improvement on that student’s assessment score compared to how they

performed on previous assessments this semester. This indicates that at least for this particular

student the study guide was useful as a study tool.

Scaffolded Study Guide

This was the most difficult aspect to analyze quantitatively. Without a doubt the most

difficult aspect of the study guide for the students was the writing components. These

components made up a large part of the study guide. The easiest aspect was the vocabulary.

This activity simply had the students looking for definitions and writing them in the study guide.

The students also did well on the cloze passages and the graphic organizers.

However, when it came time to transfer the information from the graphic organizers to

written responses the students had considerable difficulty. For every writing aspect of the study

guide we reviewed the process as a class and worked through each one together. As a result all

of the study guides have the passages completed. However, that is because I guided them

through the steps and processes they needed in order to complete them. As an example, one

section of the study guide called for the students to fill in a graphic organizer about the assistance

France gave to America during the Revolutionary War. The students completed a two-box

organizer where they list what happened before the French helped and then after the French

helped. The students were able to do these with little direction and guidance.

The next step of this process was to write a few sentences about how the French helped

the Americans. Essentially, the students needed to transcribe the notes from their organizer and

put it into paragraph form. This was a major difficulty for students across the board. So much

so that each one of the writing aspects had to be done as a whole class activity. This helps the

research from a qualitative standpoint by seeing what areas gave students the most difficulty.


However, from a quantitative standpoint the fact that most students were only able to complete

the writing aspects with assistance could skew the results. Grades were assigned on completion

and accuracy regardless of what assistance the students received.

Quantitative Analysis

When doing the quantitative analysis I analyzed all the sets of data against each other to

see if there was any correlation between various variables in the research. In all of the graphs the

students percentage score is graphed on the Y-axis and each individual student is graphed on the

X-axis. The students are ordered in every graph from low score to high score. The various

correlations are as follows.


Post Assessment and Study Guide

There was a small correlation between performance on the study guide and performance

on the post assessment. The correlation coefficient between these two sets of data is 0.35. This

is not nearly a strong enough correlation to say whether success with the study guide led to

success on the post assessment.


Post Assessment and Student Attitude

There was also a small correlation between the post assessment and students responses on

the questionnaire. The correlation between these two is 0.33. Again, this is not very significant

to conclude that there is a relationship between how the students felt about the study guide and

how they performed on the post assessment.

Fig 1. Analysis of post assessment performance and student attitude.


Study Guide and Student Attitude

There was no significant correlation between student attitude and performance on the study

guide. The correlation coefficient was .10, which is not reliable to make any assumptions

between these to aspects.

Fig 2. Analysis of study guide performance and student attitude.


Study Guide and Language Proficiency

The correlation between the student’s attitude toward the study guide and the student’s

language proficiency was 0.08. This is virtually no correlation at all.

Fig. 2 Analysis of study guide performance and language proficiency.


Student Attitude and Language Proficiency

The correlation between student attitude and language proficiency was 0.10. This is a

very weak correlation and would lead us to believe that there is no relationship between these

two variables.

Fig 3. Analysis of student attitude and language proficiency.


Post Assessment and Language Proficiency

This is the strongest correlation seen between any of the variables. However, with a

correlation coefficient of only 0.42 it is still not possible to say that there was any conclusive link

between the students post assessment scores and their language proficiency.

Fig 4. Analysis of post assessment performance and language proficiency.


Answering the Research Questions

Now that all of the data, both qualitative and quantitative, has been presented lets restate

the initial research questions and begin to answer them.

1. What are ways that we can scaffold language learning and teach content at the same time

using supplementary educational materials?

A: There are several ways that teachers can scaffold language learning. From the

qualitative analysis there is good evidence that students benefited from the cloze passages and

the graphic organizers. What is less clear is how the students benefited from the writing tasks.

What the results of this study do not show is the learning that took place when the writing

passages were done with the entire class. Through questions and observation I could see that the

students were beginning to grasp the concepts. However, with the window of this project

covering only a couple of weeks and one unit it is not possible to know if they would apply that

learning to another study guide.

2. Furthermore, how can we incorporate vital learning strategies into those materials and

activities that will help students become more efficient and self aware in their own

learning?

A: Again this is not shown in the quantitative data but there is good qualitative evidence

that the students did gain valuable tools that will help them across a variety of content areas. I

point to the organization one example of this. The fact that the students began to be responsible

for their study guides and take the work seriously is something that can be applied in any

circumstance. One benefit of doing something like a study guide that the students are forced to


keep up with for several weeks is that they learn organization. In that way this project is a

successful example of incorporating a learning strategy into curriculum.

3. Does a scaffolded study guide provided to the students for use throughout a unit provide

support for content and language while at the same time teaching learning strategies?

A: The answer to this question is at best inconclusive because if the results of the

quantitative data. The quantitative data did not reveal any strong correlations between any of the

variables including the post assessment and the study guide. This could be due to many factors

that will be discussed later. However, the data from this particular research project shows no real

quantitative evidence that the study guide aids in student learning.


Chapter 7

Discussion of Results

Research Overview

This was an action research project that used a mixed method approach to gauge the

effectiveness of a scaffolded study guide as an academic resource for English language learners.

The project was conducted over two weeks and was implemented in a sheltered instruction

American History course.

The data was gathered using qualitative methods such as observation and quantitative

methods such as numerical analysis of data. Qualitative data was gathered using an observation

protocol and category questions based on the data gathering instruments. Quantitative data was

gathered using a student questionnaire, the study guide, and a post assessment. The qualitative

data was analyzed by categorizing different pieces of information. The quantitative data was

analyzed using Microsoft excel to calculate correlation coefficients. This data was then analyzed

and used to address the research questions of the project.

Summary of Findings

The findings of this particular research study are inconclusive. There is little quantitative

evidence that there is a correlation between using a scaffolded study guide as learning tool and

actual student learning and achievement. However, there is decent qualitative evidence that

learning took place using the scaffolded study guide. This is based mainly off observations I

made during the research phase.

Solutions to the Problem

Based on the results of this research it is difficult to recommend that the scaffolded study

guide is a valid learning tool for English language learners. However, certain aspects of the


study guide such as graphic organizers, cloze passages, and vocabulary activities have been

proven to be effective methods of instruction for English language learners. The problem that

these students have is that they are at varying levels of English proficiency. Because of that the

students are processing different levels of content, English, and instruction. A higher-level

student may do better than an intermediate or beginning ELL student. The most obvious solution

to these problems to level all activities based on each students level of English proficiency.

Implications and Limitations

Strengths of The Research

Aside from the data being inconclusive this research project has several strengths. For

one I think the qualitative data gathering was done well. I was always in tune to how my

students were doing with this project. It is my view that the quantitative data does not

necessarily give a proper indication of the learning that took place with this particular

educational method. I think there is merit to the idea of using a scaffolded study guide.

However, I do think changes when implementing it into a curriculum are needed.

I think another strength of this research is that it is reliably inconclusive. I made sure to

evaluate all of the data carefully and from multiple angles. I made sure to report scores and

observations, as they were regardless of what that said about the study guide or the instruction.

This allows this project to have integrity.

Weaknesses of The Research

There are several weaknesses with this project. First, I believe that there may have been

too many tools used to gather data. I think the design of the study guide could have been simpler

and included less writing intensive activities. It was ambitious on my part to design such a

challenging study guide for the students. The students may have done better learning content


knowledge with more direct tasks in the study guide such as more cloze passages or graphic

organizers.

I definitely feel like the students learned a lot about writing while we went through this

unit. However, that is not what was measured with the post assessment. The post assessment

only measured what content concepts the students had mastered. The major weakness of this

project is that the study guide was too broad. My intent was to measure how the study guide

helped English language learners learn content. A possibly better focus may have been how did

the study help students improve their writing and knowledge of sentence structure and syntax.

Problems With the Research

The major problem with the research is that the topic was too broad. There were too

many other factors that make it hard to tell if this is an effective tool for students. I think

narrowing the research focus would have produced more reliable results. The other significant

problem with the study guide was that students did not use it during the most critical phase.

After the post assessment I asked the students how many of them used the study guide to study

for the post assessment. Only about three students raised their hands. Two of the students

achieved the top scores and one scored better than they had on any assessment all semester.

I inquired with the students why they had spent all the time and work to create a study

guide and not use it. The student responses were varied but mostly centered on "they didn’t

think about it". They just looked at it as schoolwork, something separate from the quiz. For me

it was one of those reminders that students need to learn many things not just content or English.

I truly believe many of the students did not even think about using the study guide as preparation

tool for the post assessment. When I explained to them that is what it was for they seemed to

have one of those “A-HA” moments. This realization helps me for next time but that simple fact


that many of the students did not use the study guide as a preparation leads me too believe that

the results of the entire research may be inaccurate on just that basis alone. When I look at the

students who used the study guide to study for the post assessment their scores are above

everyone else. Is that an anomaly or is that significant. I do not have enough evidence to say.

Factors That Skewed Findings

One of the most obvious factors that I think may have skewed the findings was just

described in the way the students used or did not use the study guide. The other major factor that

skewed the results of this project has to do with the student’s language ability. In this class I

have a range of students from very beginning English to advanced. This study guide was

designed as a one size fits all. In reflection it is unreasonable to assume that this would give an

indication of learning for a beginning level student.

Improvements For Future Research

I firmly believe that there is some educational value to using this study guide as a tool for

English language learners. There have been benefits that I have observed just in this project.

However, for future research I would narrow the focus of the project to specifically look at one

aspect of student learning and hone in on that. For instance, I made a post assessment to measure

content. In that case I should have designed the study guide to be more focused on content and

less on English skills and writing. I think it is possible to combine both. However, for this

project I think the study guide was too heavily weighted towards the English aspect and not so

much the content. I think careful planning in the design of the study guide and specific criteria

to be measured would make for a more reliable research project.

Also, I would suggest that in future research projects that student’s English levels be

taken into account more carefully in planning the study guide and research. I think the design of


several leveled study guides that are written for different English levels would be a better

indicator of whether this was an effective tool for learning. By doing that the teacher is meeting

the student at their level or just slightly above. This might make for enhanced learning in content

concepts and English.

Critique of Masters Degree Experience

This process has truly been an incredible learning experience, from the start of my

program until now the very end. I have enjoyed learning about English language learners and I

am truly thankful for this program because it has allowed me to have a rewarding new position in

education working with some amazing students from around the world.

This capstone process has been challenging for me but I honestly think that I have

learned more by doing this than in any other class or project I have been a part of before. The

sheer volume of information, detail, planning, and discipline that were required to complete this

project has made me a better teacher.

I will definitely use the skills that I have learned by doing this capstone in my classroom

everyday. Even though I may not write a 70 page research report on every new thing I introduce

in my classroom I will use the skills that I have learned to evaluate my curriculum and

assessments on a daily, weekly, and monthly basis. I can tell that I write more effective lessons

and create more effective assessments since I have been working on this project. This is true

across the board for all my classes. I am truly feeling an improvement in my teaching as a result

of the skills I have gained through this process.


References Abadiano, H.R., & Turner, J. (2002). Sheltered instruction: An empowerment framework

for English language learners. The New England Reading Association Journal, 38(3), 50-

55.

Borich, G.D. (2007). Effective teaching methods. Upper Saddle River, NJ: Pearson.

Brinton, D. M., Snow, M. A., & Wesche, M. (2003). Content-based second language

instruction. New York, NY: Newbury House.

Celece-Murcia, M. (2001). Teaching English as a second or foreign language. Boston,

MA: Heinle

Chamot, J.M., & O'Malley, A.U. (1987). The cognitive academic language learning

approach: A bridge to the mainstream. TESOL Quarterly, 21(2), 227-249.

Chamot, A., & O'Malley, J. M. (1994). The CALLA handbook: Implementing the

cognitive academic language learning approach. Reading, MA: Addison-Wesley

Publishing Company.

Coehlo, E. (2004). Adding English: A guide to teaching in multilingual classrooms.

Toronto, ON: Pippin.

Cline, Z., & Necochea, J. (2003). Specially designed academic instruction in English

(SDAIE): more than just good instruction. Multicultural perspectives, 5(1), 18-24.

Crandall, J. (1994). Content-centered language learning. ERIC No. ED367142,

http://www.eric.ed.gov (Accessed September 13, 2010)

Crandall, J., Jaramillo, A., Olsen, L., & Peyton, J. K. (2002). Using cognitive strategies to

develop English language and literacy. ERIC No. ED469970, http://www.eric.ed.gov

(Accessed September 13, 2010).


Echevarria, J., Vogt, M.E., & Short, D. (2008). Making content comprehensible for

English learners: The SIOP model. Third edition. Boston: Pearson Allyn and Bacon.

Echevarria, J., & Short, D.J. (2005, Summer). Teacher skills to support English language

learners. Educational Leadership, 62, 8-13.

Echevarria, J., Vogt, M.E., & Short, D. (2011). The SIOP model for teaching history-

social studies to English learners. Boston: Pearson Allyn and Bacon.

Fromkin, V., Hyams, N., & Rodman, R. (2007). An introduction to language. Boston,

MA: Thomson Wadsworth.

Freeman, D., & Freeman, Y. (1988). Sheltered English instruction. ERIC No. ED301070,

http://www.eric.ed.gov (Accessed September 13, 2010).

Gay, L.R., Mills, G.E., & Airasian, P. (2009). Educational research: Competencies for analysis and applications. Upper Saddle River, NJ: Pearson

Genesee, F., Lindholm-Leary, K., Saunders, W., & Christian, D. (2005). English

language learners in US schools: An overview of the research findings. Journal of

Education for Students Placed at Risk, 10(4), 363–385.

Hansen-Thomas, H. (2008). Sheltered instruction: Best practices for ELL's in the

mainstream. Kappa Delta Pi Record, 4(44), 165-169.

Karathanos, K. (2009). Exploring us mainstream teachers' perspectives on use of the

language in instruction with English language learner students. Journal of Bilingual

Education and Bilingualism, 12(6), 615-633.

Kidd, R., & Marquardson, B. (1994). The foresee approach to content-based ESL

instruction. Paper presented at the 28th Annual TESOL Convention. ERIC No.

ED374677, http://www.eric.ed.gov (Accessed September 13, 2010).

Lightbrown, P, & Spada, N. (2006). How Languages are learned. New York: Oxford


Meskill, C. (2005). Infusing English language learner issues throughout professional

educator curricula: The training all teachers project. Teachers College Record, 107(4),

739-756.

O'Malley, J.M., & Pierce, L.V. (1996). Authentic assessment for English language

learners. New York, NY: Longman.

Peregoy, S. & Boyle O. (2001) Reading Writing and Learning in ESL. New York:

Addison Wesley Longman

Sobul, D. (1995). Specially designed academic instruction in English. ERIC No. ED

391397, http://www.eric.ed.gov (Accessed September 13, 2010)

Tsai-Fu, T., & Wu, Y. (2010). Effects of note-taking instruction and languages in college

EFL students listening comprehension. New Horizons in Education, 58(1), 120-132.

Washburn, G. (2008). Alone, confused, and frustrated: Developing empathy and

strategies for working with English language learners. The Clearing House, 81(6), 247-

250.

Watson, S., Miller, T., Driver, J., Rutledge, V., & McAllister, D. (2005). English

language learner representation in teacher education textbooks: A null curriculm?.

Education, 126(1), 148-157.


Appendix A Field Notes: Observation Protocol: Descriptive Observations

8. What will be my role during the observation?

9. What will the students be doing?

10. What interactions if any are happening between the students?

11. What are the distractions to the learning environment?

12. How are the students using the scaffolded study guide?

13. What challenges are the students having with the scaffolded study guide?

14. How did the class end?

Reflective Observations

3. Questions arising from the observation?

4. Insights and thoughts from the observation?


Appendix B Questionnaire

Student Questionnaire Please answer the questions about the study guide that we used for this unit. There is no right or wrong answer. This is your own opinion. Place an X next to the item that best describes your level of agreement with the question.

1. This study guide was easy to use.

____ Strongly Agree ____ Agree ____ Undecided ____ Disagree ____ Strongly Disagree

2. The purpose of the study guide was clear to me.


3. The study guide made it easier to understand the material in the unit.


4. The study guide was a useful tool to have when studying for the quiz.


5. I would like to use a study guide like this for all the units.

____ Strongly Agree ____ Agree ____ Undecided ____ Disagree ____ Strongly Disagree Please add any other comments about the study guide in the space below:


Appendix C Study Guide

American Revolution Chapter 5 Study Guide Vocabulary: Use the textbook to complete the vocabulary. These words will help you understand important concepts in the chapter. Patriots - Loyalists - Loyal - Forces - Continental Army - Battles - Defeat - Excited - Idol - Valley Forge - Miltary Officers - Official - Alliance - Manual - Prisoners of War - Fleet - Peninsula - Outnumbered - Treaty-


Chapter Outline: Complete the study outline for chapter 5. The American Revolution

I. Two Very Different Armies

A.

B. The Battle Saratoga - A Turning Point

Notes:

II.

A. Winter at Valley Forge

B.

C.

Notes:

III.

A.

B.

Notes:


Chronological Order: Place the events in the correct order that they occured on the

timeline. Make sure to include the year.

1. Americans Lose Many Battles

2. The British Surrender at Yorktown

3. Americans Won The Battle of Saratoga

4. The French Signed an Alliance with the Americans

5. Many Soldiers Died at Valley Forge

6. The British Won The Battle of Charleston

Using Language: Use the events above to write about the American Revolution.

Include at least two dates and use the signal words after, next, then, first, meanwhile,

and soon.

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________


Key Facts: Complete the sentences with your vocabulary words

1. The __________________ did not fight against the British.

2. George Washington was a ____________________.

3. The ___________________ wanted independence from Britain.

4. The Americans and the British signed a _______________ that gave the colonies

independence.

5. After the war, the British freed their _____________________________.

Organizing Information: Use the information in Chapter 5 to complete this web.

British Generals:

The American Revolution

American Generals:

When: Battle that ended the war:


Before and After:

Use the information in Chapter 5 to complete the chart.

Before the French Helped After the French Helped

Interpreting Information: Use the information above to write about how the French

changed the American Revolution.

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

________________________________________


Chapter Reflections:

What Did I Learn:

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

What is Still Unclear:

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

______________________________________________________________________

My Favorite Part of This Chapter Was:

______________________________________________________________________

______________________________________________________________________

____________________________________________________


Appendix D Post Assessment

Chapter 5 Quiz Multiple Choice: Place an (X) next to the best answer. There is only one correct answer. 1. The Americans and the French made an alliance. What does the word alliance mean? ____ A small book of instructions ____ An early surrender to an army ____ A large group of soldiers ____ An agreement between two countries 2. How was the Continental Army different from the British Army? ____ It did not have many guns and uniforms ____ It was fighting far from home ____ It was well trained to follow orders ____ It did not have a good leader 3. What happened just after the Battle of Saratoga? ____ King George III surrendered ____ France agreed to help America ____ General Cornwallis took control of the British Army ____ George Washington became president


4.Why did the British move the war south in 1778? ____ More patriots lived in the south. ____ They had defeated the patriots in the north ____ More loyalists lived in the south ____ The largest cities were in the south 5. Who was the leader of the Continental Army? ____ Charles Cornwallis ____ George Washington ____ Marquis de Lafayette ____ Benjamin Franklin 6. During the winter of 1777-1778 the Continental Army was encamped at? ____ Valley Forge ____ Philadelphia ____ Charleston ____ New York City 7. What happened as a result of the Treaty of Paris? ____ America became a new nation ____ France agreed to help the Americans ____ America declared war on Britain ____ Britain took control of Georgia


8. The war ended with the British surrender at? ____ Valley Forge ____ Yorktown ____ Saratoga ____ Charleston 9. The British soldiers were known as? ____ Loyalists ____ Patriots ____ Redcoats ____ Bluecoats 10. To place this in the order that they happened means to put them in

_______________ order. ____ Reverse ____ Split ____ Timed ____ Chronological


Appendix E Scoring Rubric for Study Guide


Appendix F Qualitative Study Guide Questions

1. What sections gave students the most trouble? 2. Do student reflections indicate learning? 3. Were there sections where completion is inconsistent with directions? 4. Are there any additional things the student has added to the study guide?


Appendix G Informed Consent Form

INFORMED CONSENT FORM

Western Governors University

Masters of Arts English Language Learning Language and Content Learning Using a Scaffolded Study Guide

Anthony McConnell

Introduction Students in 5th period ELL American History are invited to participate in a research project being conducted by their teacher, Anthony McConnell as partial fulfillment of the requirements for the Masters of Arts in English Language Learning from Western Governors University. I am conducting this research to explore the most optimal ways to combine language and content learning. Description of the project:

• This research will be conducted over a two-week unit about the revolutionary war. • At the beginning of the unit students will be given a scaffolded study guide to use

throughout the unit. This study guide consists of vocabulary activities, graphic organizers, cloze sentences and writing prompts.

• The research will investigate whether or not the study guide is a useful tool to supplement learning in ELL content courses.

• This research will take place 5th period in room 123 of Rock Island High School during the regular class session. This research project will last approximately two weeks beginning on October 18th and ending on October 29th.

• This research will in no way interfere with regular classroom activities, rules, or regulations. All students are expected to participate fully in all regular classroom activities.

• Along with regular classroom expectations all students will complete the scaffolded study guide throughout the unit, they will also complete a post assessment, and a short questionnaire about their experience using the scaffolded study guide.

Benefits and Risks of this study: This project has many potential benefits. The study guide can provide students with additional ways to learn content material and language. It offers ongoing support throughout the unit. Should the study guide prove to be an effective educational tool it could be used in other units in class. Furthermore, when proven a successful tool it can be shared with other teachers and administrators in other departments, schools, and school districts. There are no risks associated with this study. Students will not be asked to do anything beyond regular class expectations.


Confidentiality: The confidentiality of each participant will be strictly maintained, records will only be seen by myself, and all data that is reported will be aggregated. Voluntary participation and withdrawal:

• Participants are expected to participate in all regular classroom instruction. Few aspects of this research goes outside the bounds of regular classroom instruction. Students will be receiving a classroom grade for the scaffolded study guide and the post assessment that will influence their semester grade.

• Students may choose to not participate in the questionnaire. There will be no penalty associated

with this should they choose not to participate.

• At no time will the students be videotaped, photographed, or audio recorded.

• Individual participants may request that their results be eliminated from final reporting. Questions, Rights and Complaints: If you have any questions you may contact me. Participants and their legal guardians have a right to the results of the study. Please contact me after 11/22/2010 is you would like a summary of the results.

Consent statement: By signing this consent form the participants and legal guardians agree to participate in this study and confirm that they have been informed of the purpose, benefits, and risks associated with participation. ________________________ _______________________ Signature of Participant Signature of Legal Guardian (Student) _________________________ _______________________ Typed/printed Name Typed/printed Name __________________________ ________________________ Date Date

Capstone Final

Documents

Transcript of Capstone Final