Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation...

50
Revised 2/5/07 5:29 PM 1 "Classrooms for the Future" Evaluation Plan for Year One Educational leaders are actively looking for new approaches that make high school more engaging, more effective, and more likely to create graduates who are able to compete in a global marketplace. One new approach to high school reform is Pennsylvania's "Classrooms for the Future" (CFF) initiative. In his 2006 budget address, Pennsylvania Governor Ed Rendell announced that the CFF initiative, will place a laptop computer on every student desk in every public high school classroom in which the four core subjects are taught by 2009. 1 The initiative will also equip each of these classrooms with a multimedia teaching station, provide online professional development for educators, and position a "Coach" in every high school to encourage rapid adoption of effective technology-based teaching practices. The purpose of the CFF initiative is to transform Pennsylvania's high schools, making them more engaging and more responsive to economic threats presented by globalization. The evaluation of the CFF initiative is a partnership among The Pennsylvania Department of Education (PDE), the Regional Educational Lab for the Mid-Atlantic region, and CaseNex, a professional development company that emerged from work at the University of Virginia's Curry School of Education. These three partners worked together in the design of the evaluation. PDE and CaseNex will work with the Penn State Survey Research Center to gather the data, and then the data will be analyzed and described by the Regional Educational Lab. The focus of the assessment in Year One is formative assessment – gathering data that can be used to guide implementation. In years two and three, the evaluation will be expanded to examine the effects of the CFF initiative on student achievement and the development of "21 st Century Skills." This document describes the approach to the Year One data collection in the initial 103 high schools implementing the initiative late in the 2006-2007 school year. Questions or Key Issues to be Addressed The key questions guiding this project were first generated in discussions moderated by REL Mid-Atlantic personnel, attended by key bureau administrators at the Pennsylvania Department of Education representing PreK-12 Education, Curriculum and Instruction, High School Reform, and Instructional Technology. A draft of the questions that emerged was compiled and sent to PDE and to REL PIs for comment. These research questions, contained in the matrix presented as Appendix A, focus on five areas: the impact of the CFF initiative on teaching practice; the impact of the CFF initiative on student activity; the impact of the CFF initiative on teacher attitudes; the impact of the CFF initiative on student attitudes, and; factors that may be enhancing or limiting the initiative's impact. In future years PDE and the Pennsylvania legislature that is providing the funding for the initiative will also be interested in assessing the impact of the initiative on student achievement, but student achievement is not a factor under investigation in this initial year. These research questions are important to the Pennsylvania Department of Education because they will provide the data that will be needed to steer this very expensive, potentially important project. The state will invest over $200 million in the technologies alone, and hundreds of millions more in time and energy. Technologies alone will not change education, but teachers using the technologies effectively may. The answers to these questions will let the leaders of this potentially important initiative to determine whether they are, in fact, changing teacher and student activity, which is a prerequisite to having impact on student achievement and drop out rates. Methodology This is a descriptive study, using data that is being collected by the Pennsylvania Department of Education. Because the Year One CFF schools were selected through a competitive process, they are not necessarily representative of the schools not yet implementing the program. For this reason, and because the purpose of the evaluation is to identify changes that take place in CFF schools as a result of 1 A video of the segment of the Governor's speech dealing with CFF can be seen at http://1to1.ed.psu.edu/conference/1to1%20Conference%202006/1-to-1%20Proposal%20for%20PA%20High%20Schools.html

Transcript of Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation...

Page 1: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

1

"Classrooms for the Future" Evaluation Plan for Year One

Educational leaders are actively looking for new approaches that make high school more engaging, more effective, and more likely to create graduates who are able to compete in a global marketplace. One new approach to high school reform is Pennsylvania's "Classrooms for the Future" (CFF) initiative. In his 2006 budget address, Pennsylvania Governor Ed Rendell announced that the CFF initiative, will place a laptop computer on every student desk in every public high school classroom in which the four core subjects are taught by 2009.1 The initiative will also equip each of these classrooms with a multimedia teaching station, provide online professional development for educators, and position a "Coach" in every high school to encourage rapid adoption of effective technology-based teaching practices. The purpose of the CFF initiative is to transform Pennsylvania's high schools, making them more engaging and more responsive to economic threats presented by globalization. The evaluation of the CFF initiative is a partnership among The Pennsylvania Department of Education (PDE), the Regional Educational Lab for the Mid-Atlantic region, and CaseNex, a professional development company that emerged from work at the University of Virginia's Curry School of Education. These three partners worked together in the design of the evaluation. PDE and CaseNex will work with the Penn State Survey Research Center to gather the data, and then the data will be analyzed and described by the Regional Educational Lab. The focus of the assessment in Year One is formative assessment – gathering data that can be used to guide implementation. In years two and three, the evaluation will be expanded to examine the effects of the CFF initiative on student achievement and the development of "21st Century Skills." This document describes the approach to the Year One data collection in the initial 103 high schools implementing the initiative late in the 2006-2007 school year. Questions or Key Issues to be Addressed The key questions guiding this project were first generated in discussions moderated by REL Mid-Atlantic personnel, attended by key bureau administrators at the Pennsylvania Department of Education representing PreK-12 Education, Curriculum and Instruction, High School Reform, and Instructional Technology. A draft of the questions that emerged was compiled and sent to PDE and to REL PIs for comment. These research questions, contained in the matrix presented as Appendix A, focus on five areas: the impact of the CFF initiative on teaching practice; the impact of the CFF initiative on student activity; the impact of the CFF initiative on teacher attitudes; the impact of the CFF initiative on student attitudes, and; factors that may be enhancing or limiting the initiative's impact. In future years PDE and the Pennsylvania legislature that is providing the funding for the initiative will also be interested in assessing the impact of the initiative on student achievement, but student achievement is not a factor under investigation in this initial year. These research questions are important to the Pennsylvania Department of Education because they will provide the data that will be needed to steer this very expensive, potentially important project. The state will invest over $200 million in the technologies alone, and hundreds of millions more in time and energy. Technologies alone will not change education, but teachers using the technologies effectively may. The answers to these questions will let the leaders of this potentially important initiative to determine whether they are, in fact, changing teacher and student activity, which is a prerequisite to having impact on student achievement and drop out rates. Methodology This is a descriptive study, using data that is being collected by the Pennsylvania Department of Education. Because the Year One CFF schools were selected through a competitive process, they are not necessarily representative of the schools not yet implementing the program. For this reason, and because the purpose of the evaluation is to identify changes that take place in CFF schools as a result of

1 A video of the segment of the Governor's speech dealing with CFF can be seen at http://1to1.ed.psu.edu/conference/1to1%20Conference%202006/1-to-1%20Proposal%20for%20PA%20High%20Schools.html

Page 2: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

2

the initiative, the project will examine changes from baseline levels rather than employing control group comparisons. For most research questions, data will be collected from multiple sources, as illustrated in the matrix presented as Appendix A. Data sources are listed (in the order the data will be collected) in the table below, and each is described briefly on the following pages. Data Source Administered Pennsylvania Assessment of Technology Integration (PATI) (Appendix E)

Fall, 2006 (baseline)

CFF Classroom Observation Tool (Appendix D)

Feb 2 – Mar 30, of 4 teachers per CFF School (baseline) May 7 – June 1, same 4 teachers (post)

Teaching in Pennsylvania Record (Observation) (Appendix F)

Feb 2 – Mar 30, two observations each of 4 teachers per CFF School (baseline) May 7 – June 1, two observations each of same 4 teachers (post)

CFF Teacher Surveys (Appendix B)

Mar 19 – Mar 30 by all CFF Teachers (baseline) May 21 – June 1 by all CFF Teachers (post)

CFF Student Surveys (Appendix C)

Mar 19 – Mar 30 by one class of all CFF Teachers (baseline) May 21 – June 1 by the same class of all CFF Teachers (post)

Coach Survey May 21 – June 1 Administrator Survey May 21 – June 1 Participants in Surveys and Observations The CFF Teacher Surveys will survey all teachers teaching in CFF classrooms (approximately 700 teachers). The two observations, however, will be conducted on a subset of the CFF teacher population consisting of two randomly selected math and two randomly selected language arts teachers from each CFF school (approximately 412 teachers). To identify the random teachers for observation, the Pennsylvania Department of Education, with assistance from REL Mid-Atlantic will:

• place information on all CFF teachers in an excel spreadsheet • sort the data by school • use the RANDOM function in Excel to assign a random number of each teacher • copy and paste that column as text (so that the random numbers will not change) • sort the spreadsheet by the random numbers within schools, and • select the first two math and the first two language arts teachers for observation • inform the data collectors of the teacher names.

The data collectors will assign all CFF teachers a random "teacher code" using a list of 100 random codes supplied to them through the CaseNex website established for the CFF initiative. To ensure confidentiality, the list of names and codes will be maintained at the school, and not forwarded to the research team nor PDE. The data collectors will inform the PDE of the four teacher codes associated with the teachers who will be observed, so that PDE can monitor completion of data collection. All CFF teachers will use the teacher codes when completing surveys, so that pre and post data can be compared, and one class of each teacher's students will enter the same teacher's code during their surveys so that the student data may be associated with the teacher.

Page 3: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

The Instruments Each of the data collection instruments that will be used in the Year One CFF Evaluation is described below. Pennsylvania Assessment of Technology Integration (PATI) Designed by the Metiri Group for the Pennsylvania Department of Education, this tool is derived from instruments created by the Metiri Group for the State Education Technology Directors Association (SETDA). This survey is based very closely on the SETDA Teacher Survey. In four rounds of field tests, the Metiri research team conducted a multistage review of the survey items and scales. There were two main goals for this process: assuring that the survey was as reliable as possible, and reducing the overall number of items to the extent possible while maintaining sufficient reliability and validity in the overall survey set. A third outcome of the item review process was a final validity check on the items within each scale. The statistics and processes used were used to guide rather than dictate the changes that were made. While seeking the highest technical indicators of reliability, the statistics were used to identify issues or problems, with the Metiri research team deciding whether to delete, revise, consolidate, or keep items intact. Final survey changes were driven by a combination of statistical inference, framework fidelity, and the experience of the Metiri team. The PATI is required of all public school teachers in Pennsylvania (approximately 107,000) and is taken in the Fall of each year. Several items from the PATI survey (listed in Appendix E) are directly related to research questions for this project, and will be used as baseline information, since the PATI survey was administered well before the CFF initiative was implemented. The CFF Classroom Observation Protocol is also based upon a protocol developed for the SETDA suite of tools. This protocol was developed in concert with the survey instrument described above and was used in the same four phases of field trials. The tool has been used in projects nationwide and has been the subject of several content validity reviews and rater calibration processes. This observation protocol will be administered by trained observers, generally building principals in the CFF schools. Each CFF school was asked to identify an observer, and PDE recommended that the Principal be designated to fill this role. A seven-hour training session was provided for this protocol and the TPR (described below) in three locations across Pennsylvania, in Pittsburgh (January 23, 2007), Harrisburg (January 24, 2007) and Reading (January 25, 2007). As is also the case for the TPR, described below, a computer-based version of this CFF Observation Tool will be downloaded by the trained observers from the "Downloads" window on the CaseNex CFF website (see illustration below), after logging in at http://tpr.casenex.com/).

Revised 2/5/07 5:29 PM

3

Page 4: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Whether run on a Windows-based computer, an Apple Macintosh, or a Dell Axiom (provided to the data collectors by the CFF project, the data collection tool will look identical, because the computer-based versions have been designed to look like the Dell Axiom as illustrated below.

The Teaching Performance Record (TPR) The TRP is a "research-based" observation protocol developed at the University of Virginia's Curry School of Education, and disseminated by "Casenex" a company UVA created to continue the development of and promote the use of the TPR. Schools use this observation tool and a series of case study-based professional development resources in the belief that "a strategic view of teaching advances the use of instructional strategies or models to plan, teach, and assess teaching (Eggen & Kauchek, 2006; Gunter, Estes, & Schwab, 1999; Joyce, Weil, & Calhoun, 2004)." Casenex reports that:

"Empirical evidence supports the contention that Strategic Teaching influences middle school student learning in mathematics (Konold, Jablonski, Nottingham, Kessler, Byrd, Imig, Berry, & McNergney, 2006). There is no general consensus, however, on the combination of teaching behaviors that are best for all students and all objectives. The assumption is that an effective teacher is one who can plan and teach different ways in order to meet the needs diverse learners who must accomplish a variety of outcomes. Teachers who possess a rich repertoire can call up appropriate strategies when conditions warrant."

Working from that base, the TPR consists of 111 items that combined to produce five "attributes of Strategic Teaching, including: Focus/Capacity (17 items), Syntax (17 items), Principles of Reaction (27 items), Social System (25 items), and Evaluation (25 items)."

As is the case with the CFF Classroom Observation Tool, the data collectors were trained to use the TPR in face-to-face training sessions conducted by Casenex, in January of 2007. In addition to the training, observers are required to practice using the tool with online videotaped teaching segments and then must pass a test on observing skills by reaching 75% agreement with expert reviewers' ratings of the test video segments.

The TRP will be used to observe the same four randomly selected teachers, and will be used twice during the baseline period and twice during the post assessment period with each of the four selected teachers.

Revised 2/5/07 5:29 PM

4

Page 5: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

5

The CFF Teacher Survey and CFF Student Survey These surveys were developed as modifications of other surveys developed by the Metiri Group for the State Educational Technology Directors' Association (SETDA). The items have been used in surveys with thousands of respondents, and were developed, piloted and revised over years of use. We will not be assembling items to form scales, but rather will be looking at aggregated responses to individual items that address research questions, as indicated in the matrix presented as Appendix A. These surveys will be converted to online format and hosted by the Penn State Survey Research Center, because of their experience and ability with conducting large-scale online surveys. In year one we will survey approximately 700 teachers (All CFF Teachers) and approximately 17,500 students (one class of students for each CFF teacher). Coach and Building Leader Surveys The Coach and Building Leader surveys have not yet been developed by PDE, but will be variations of the Teacher's survey, administered to the CFF Coaches (technology integration professional development experts) and the building leaders (principals) in June, as a post-only look at their perspectives. This information will be used as to help interpret data received from teachers and students, but is not critical to the success of the project. The building leader surveys may be administered as telephone interviews, but that has not yet been determined. Data Analysis and Reporting The primary purpose of this evaluation is to document the changes that occur during the initial months of implementation of the CFF high school reform project. The report will be descriptive in nature, stating each research question in turn, displaying the data gathered from baseline and post implementation assessments, and describing the differences. We intend to use Chi squared analyses to assess the significance of differences between baseline and post intervention data on all categorical survey and observation items (the vast majority). Some items on the observation tools ask the respondent to indicate a percentage rather than selecting a category associated with a percentage change. For these items we intend to calculate and display mean percentages across observations, and use a t-test to determine whether the baseline/post differences are significant. The .05 level will be used as indicate statistical significance for all research questions.

Year One Evaluation Timeline: PATI – Pa Technology Inventory Fall 2006 Casenex – TPR – Create Login and Teacher Code numbers (create for every teacher and communicate to them, they will need for the teacher survey)

January 23-25

Practice using TPR January 29-February 2 Take progress test on TPR Complete by February 2 Collect TPR Observation Data (pre) (4 teachers twice) February 2 – March 30 Collect CFF Observation Data (pre) (4 teachers once) February 2 – March 30 Teacher Survey (pre) – Use teacher code March 19 – March 30 Student Survey (pre) – Use teacher code March 19 – March 30 Collect TPR Observation Data (post) (4 teachers twice) May 7 – June 1 Collect CFF Observation Data (post) (4 teachers once) May 7 – June 1 Teacher Survey (post) – Use teacher code May 21 – June 1 Student Survey (post) – Use teacher code May 21 – June 1 Post Administrator Interview May 21 – June 1 Post Coach Interview May 21 – June 1 In Year 2, we anticipate the following milestones: Pre Data Collection September 2007 Mid Year Data Collection January 2007 Post Data Collection May 2007

Page 6: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

6

Appendix A

Research Questions and Data Sources

Page 7: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

7

PDE Interview(s)

CFF Websites

PDE/Govt Docume

/

Implemen-tatio T

S SurPATI urve

CFF srova

CaseNex TPR setio

Data from

Sch

CFF oach

ys

er

doloistia

Research Question nts

Videos Ren

ports eacher urveys

Student veys S y

ClasObser

om tions

Ob rva-ns

ools C

Surve

BuildingLeadSurveys

Metho/ StatAppro

gy cal ch

A. Project Overview

1. Why & how was CFF created? X X X

Inerviews & cument

Analysis Do

2. What is CFF? (program description) X X X

Inerviews & cument

Analysis Do

3. What is the timetable for i

X X X Inerviews &

cument Analysis mplementation?

Do

B. Implementation

1. What is the status of implementation a

X

Descriptive (reporting

% of c ssrooms

up and running) s of March 1?

la

2. What is the status of implementation a

X

Descriptive (reporting

% of c ssrooms

up and running) s of June 1?

la

3. What are the primary causes of i

X

Descriptive (Frequencies

and Narrative) mplementation delays?

C. Impact on Teaching Practice

1. Has CFF changed classroom layout (arrangement of desks, etc.)?

Q1 Q2 Pre vs Post Chi Squared

2. Has CFF changed the percentage of t

Q2a - Q2f

Q2 Q3 Pre vs Post

ime teachers spend lecturing and in other activities?

Chi Squared

T-test

3. Has CFF changed the "complexity" of c

Q3 Q3 Q15 Q5, 10,

15

Pre vs Post

lass content, moving from "basic skills" to more "higher-order" topics?

Chi Squared T-test

4. Has CFF changed the "instructional s

Q4 Q4 Q6, 11,

16

Pre vs Post

tyle" exhibited by the teacher, on a scale from "didactic" to "constructivist?"

Chi Squared T-test

5. Has CFF changed the "relevance" of c

Q5 Q7, 2,17

Pre vs Post

lass content, moving from "artificial" to more "real world" in nature.

1

Chi Squared

T-test

Page 8: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

8

6. What technologies are being used in t

Q8a - Q8r

, Q24 Pre vs Post

he different subject areas, and how is this changing as a result of CFF?

Q21

Chi Squared

7. Are teachers experiencing any c Q14 19 Pre vs Post hanges in classroom management

(discipline problems, time lost in transitions, etc.) as a result of CFF?

h Q Chi Squared

8. Are teachers comfortable teaching in C

Q14 Q29c Q20 Pre vs Post

FF classrooms? i

Chi Squared

9. What different software applications are being used in CFF classrooms?

Q8a - Q8o

Q11a - Q11l

Q19 2 Pre vs Post

Q 2 Chi Squared

10. What instructional strategies are u

Q9a

throu Q9l

3 Pre vs Post sed in CFF classrooms?

gh Q2 Chi Squared

11. Do the instructional strategies used

ains?

X Pre vs Post by CFF teachers appear to change in ways supported by the TPR's five research-based dom

Chi Squared

12. Is there a difference in the attention p Q6 Q1 Q27 Pre vs Post aid to "21st Century Skills" in CFF

classrooms?

Chi Squared

D. Impact on Student Activity

1. Has CFF changed the level of student e

Q7 Q5 Q4, Q9,

Q14

Pre vs Post

ngagement?

Chi Squared

T-test

2. Has CFF changed percentages of time students are listening, engaged in i

Q2a -

Q2fQ10b,d,endependent work, and working in groups

or teams?

Q6 26 Pre vs Post

Q

Chi Squared T-test

3. In CFF classrooms, how much time do students spend using computers?

Q10 Q6h Q11, 12

25 Pre vs Post g Q Chi Squared

4. Does CFF change the type of products on which students' grades are based?

Q11a

throu Q11g

Q7 8 Pre vs Post gh Q1 Chi Squared

5. Does CFF affect student attendance? X -test T

6. Does CFF affect the number of d

X -test iscipline referrals?

T

E. Impact on Teacher Attitudes

Page 9: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

9

1. Do teachers' opinions about the potential value of technology in their c

Q8r, Q12lassrooms change during their CFF

experience?

Pre vs Post Chi Squared

2. Do teachers believe that the quality of t

Q13 Pre vs Post

he learning experience they offer has increased as a result of CFF?

X Chi Squared

3. Do teachers believe student learning has increased as a result of CFF?

Q14 Pre vs Post a X Chi Squared

4. Does CFF change the type of work t

Q14b Pre vs Post

hey assign and the quality of work they expect from students?

Chi Squared

5. Do teachers develop more t

Q14c, Q14

Q24, Q28a

28i Pre vs Post

echnology-related skills and feel better prepared to teach using technology?

g -

Chi Squared

6. Do teachers believe that they are w

Q14d, Q14

Pre vs Post orking longer or harder as a result of

CFF?

e X

Chi Squared

7. Does CFF change the way teachers feel about teaching as a profession?

Q14 Pre vs Post f Chi Squared

8. How important do teachers believe t Q15 Pre vs Post he CFF Coaches to be in the success of

CFF? i X

Chi Squared

9. Which of the CFF Coach r

Q15a - Q15h

Pre vs Post esponsibilities do the teachers perceive

as most important? X

Chi Squared

10. How important do teachers believe t Q16 Pre vs Post he building Principal has been in the

success of CFF? f

Chi Squared

11. Which of the Building Principal's c

Q16a - Q16

Pre vs Post

ontributions do the teachers perceive as most important?

e

Chi Squared

F. Impact on Student Attitudes

1. Do students believe that the quality of t

Q1 Q8o

Pre vs Post

he learning experience offered has increased as a result of CFF?

Chi Squared

2. Do students' opinions about the value o

Q8 b,c,d

Pre vs Post f school change as a result of their CFF

experience?

Chi Squared

Page 10: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

10

3. Does student interest in math, s

Q9 Pre vs Post

cience, language arts, and social studies increase as a result of CFF?

Chi Squared

4. Does CFF influence students' p

Q8

f,g,h Pre vs Post

erceptions of their preparation for college and for life after school?

Chi Squared

5. Does CFF influence students' interest in

Q8 i, Pre vs Post

teaching as a career? j

Chi Squared

6. Do students believe that they are w

Q8 k,l,m,orking longer or harder as a result of

CFF?

n Pre vs Post

Chi Squared

G. Factors that May Be Enhancing or Limiting Program Impact

1. What technology-related problems are e

Q17 X Pre vs Post

xperienced that might be limiting the program's impact?

X Chi Squared

a) Hardware failures Q10a Q35d X Pre vs Post X Chi Squared

b) Network Downtime Q10b Q35e X Pre vs Post X Chi Squared

2. What "People" or "System" Problems might be limiting the program's impact?

X X D scriptive e

a) Theft X D scriptive X e

b) Vandalism X D scriptive X e

c) Lost computers X D scriptive X e

d) Tech support Q10c Q35i X Pre vs Post X Chi Squared

e) Professional Development X D scriptive X e

f) The CFF Coaches Q15 Q35j Pre vs Post X Chi Squared

g) Leadership Q16 Pre vs Post X Chi Squared

Page 11: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

11

Appendix B

Classrooms for the Future

Teacher Survey

Page 12: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

CFF Teacher Survey (Twice a year, Pre/Post, as an online survey)

Teacher's Code: ___________________________ Class (Choose One):

Literature

Writing

Other Language Arts

Algebra

Calculus

General Math

Geometry

Trigonometry

Other Math

Chemistry

Earth Science

General Science

Physics

Other Science

Civics

History

World Cultures

Other Social Studies

Revised 2/5/07 5:29 PM

12

Page 13: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

13

1) Which of the following best describes the way your classroom is arranged?

a. Desks in rows

b. Small clusters of student desks

c. Student desks in circles or semi-circles

d. Desks along classroom walls

e. Other

2) Estimate the percentage of time you spend in each activity.

None 1% to 15% 16% to 30% 31% to 50% More than 50%

a) Whole group lecture/instruction

b) Leading a whole class discussion

c) Working with small groups of students

d) Working with individual students

e) Walking, observing, and interacting with students

f) Working at your desk, or working on other work not involving students

3) How would you describe the content your class is designed to convey?

a. Almost all content knowledge

b. More content knowledge than higher order skills

c. Half content knowledge and half higher order skills

d. More higher order skills than content knowledge

e. Almost all higher order skills

4) How would you describe your teaching style?

a. Didactic – providing instruction and direction

b. More didactic than constructivist

c. An even balance of didactic and constructivist

d. More constructivist than didactic

e. Constructivist – guiding students as they build understanding

Page 14: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

14

5) How would you describe the work that students do?

a. Exercises and assignments submitted to teacher only b. A blend of "a" and "e," but more like "a" than "e" c. A balance between "a" and "e" d. A blend of "a" and "e," but more like "e" than "a" e. Projects and products that resemble what people do outside of

school 6) What percentage of your class time would you say is directed to the development of

"21st Century skills," such as creativity, problem solving, teamwork, critical thinking, and multimedia communication skills?

a. None (0%) b. 1% to 25% c. 25% to 50% d. 50% to 75% e. More than 75%

7) During a typical class, what percentage of your students would you say are "actively

engaged" in the class (paying close attention and doing what the teacher has asked)?

a. 0 to 20% b. 21% to 40% c. 41% to 60% d. 60% to 80% e. More than 80%

Page 15: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

15

8) Please indicate how valuable you believe each computer application listed below is in your classroom. If you don't use an application, choose "Don't Use" instead.

Don't Use Not Valuable Little value Valuable Very Valuable

a) Educational management software (attendance, grades, lesson plans, etc.)

b) Drill & practice software, Integrated learning systems and/or educational games

c) Word processing

d) Data management (spreadsheets), graphing, or analysis software (EXCEL, SPSS, STATVIEW, etc.)

e) Database software (FileMaker Pro, Microsoft Access)

f) Presentation Software (PowerPoint, Keynote, Profcast, etc.)

g) Email

h) Other communication tools (Instant messaging, iChat, discussion boards, video conferencing, etc.)

i) Desktop Publishing Software (PageMaker, Pages, Microsoft Publisher, etc.)

Page 16: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

16

Don't Use Not

Valuable Little value Valuable Very Valuable

j) Web publishing software (Dreamweaver, iWeb, etc)

k) Internet for research

l) Multimedia Reference Tools (e.g. CD ROMs or online encyclopedia)

m) Simulations / Modeling Software

n) Video, graphics, or sound editing or production software (iTunes, Movie Maker, iMovie, iDVD, Garage Band, etc.)

o) "Probeware" (input devices for gathering data)

p) Web based digital curriculum or curriculum resources

q) Other Web-based learning activities

r) Overall, how valuable are the technologies currently in your classroom?

Page 17: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

17

9) Please indicate how valuable you believe each instructional strategy listed below is in your classroom. If you don't use an application, choose "Don't Use" instead.

Don't Use Not Valuable Little value Valuable Very Valuable

a) Teacher lecture or demonstration

b) Teacher-led discussion aimed at factual, knowledge-level content.

c) . Teacher-led discussion aimed at high-level outcomes (synthesis, evaluation, integration, application, etc.)

d) Project based learning

e) Problem based learning

f) Authentic learning (learning tied to the world outside the classroom)

g) Multi-modal teaching (ie, visual, auditory, kinesthetic)

h) Peer teaching or peer tutoring

i) Collaborative learning – informal (no formal roles assigned)

j) Collaborative learning with formal roles

k) WebQuests

l) Learning centers

Page 18: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

18

10) Please estimate the percentage of time a student typically spends in each of the following activities in a typical week of classes.

0% to 20% 21% to 40% 41% to 60% 61% to 80% More than 80%

a) Listening to the teacher (in a large group setting)

b) Listening to other students (in a large group setting)

c) Working independently

d) Working in groups

e) Talking with the teacher in 1-to-1 or small group conversations

f) Off Task (not doing what the teacher intended)

g) Using Computers

11) Please indicate the percentage of a student's grade that is based on each of the following

assessment methods.

None (0%) 1% to 25% 25% to 50% 51% to 75% More than 75%

a) Tests and Quizzes

b) Papers and Written Reports

c) Working independently

d) Oral Reports and Presentations

e) Projects

f) Class Participation

g) Work produced by a group or team, rather than individually

Page 19: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

19

12) When we look back on the progress made in teaching your subject ten years from now, the contribution of technologies will be seen as:

a. Minimal (There will be little evidence that technologies have helped in trivial ways, and their future in education will be in question.)

b. Marginal (There will be evidence that technologies have helped, but not in a way that justifies the expense.)

c. Moderate (Technologies will have made an important contribution, but the balance between cost and benefit will make the future of technologies in education questionable.)

d. Significant (Technologies will have had an important impact on teaching and learning and will continue to be used in schools that can afford them).

e. Very Significant (Technologies will have made very valuable contributions that demonstrate value that is greater than the cost, and 1-to-1 student/computer ratios will become the rule rather than the exception.)

13) Given the tools and resources available to me, the learning experience I can offer

students in my classroom is:

a. Poor b. OK c. Good d. Very good e. Excellent

Page 20: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

20

14. For each of the following statements, indicate your level of agreement by selecting the response that best describes how you feel.

Strongly

Disagree Disagree Somewhat

Disagree Somewhat

Agree Agree Strongly

Agree a) Students learn a lot

in my classroom.

b) I can expect work of the highest quality from our students.

c) I feel better prepared to teach than I did last year.

d) I am working HARDER than I have in past years.

e) I am working LONGER than I have in past years.

f) I would recommend teaching to a friend considering entering the profession

g) I have the technology skills I need to teach my subject using the best methods available.

h) Students in my classroom are generally well behaved and do what they are asked to do

i) I feel comfortable and confident while teaching.

Page 21: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

21

15) For the items below, please indicate how valuable each of the various activities performed by the Classrooms for the Future Coaches have been to you. If your coach does not provide a particular type of service, select "N/A."

N/A Not Valuable Little value Valuable Very Valuable

a) Teaching me to operate computers, networks, or software programs

b) Suggesting ways to incorporate technology to teach the content in my classes

c) Teaching demonstration lessons in my classroom

d) Solving technical problems (Printer won't print, network is down, etc.)

e) Helping me think about how to assess technology-rich lessons

f) Observing my instruction and providing feedback

g) Advising me on how to use technology through a differentiated instruction approach to meet individual student needs.

h) Leading CFF-related professional development workshops

i) Overall, how important is the CFF Coach to the success of your school's CFF effort?

Page 22: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

22

16) For the items below, please indicate how valuable each of the various activities performed by your building Principal have been to you. If your principal does not provide a particular type of support, select "N/A."

N/A Not Valuable Little value Valuable Very Valuable

a) Suggesting ways to incorporate technology to teach the content in my classes

b) Teaching demonstration lessons in my classroom

c) Helping me think about how to assess technology-rich lessons

d) Observing my instruction and providing feedback

e) Advising me on how to use technology through a differentiated instruction approach to meet individual student needs.

f) Overall, how important has your building principal been to the success of your school's CFF effort?

Page 23: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

23

17) For each of the potential problems listed below, please indicate the degree to which this has interfered with teaching and learning in your classroom.

No Problem

Small Problem Problem Significant

Problem Huge

Problem a) Computer failures b) Network Downtime c) Battery Issues d) Smart board problems e) Projector Problems f) Sound Problems g) Printing Problems. h) Theft i) Vandalism k) Plagiarism l) Lack of Technical

Support

m) Inadequate professional development

Thank you VERY MUCH

for taking the time to provide this important information.

Your responses have been recorded, and will be combined with the responses of other CFF teachers to provide important information with which to guide the CFF program.

If you would like to provide additional information or comments, please CLICK HERE to do so.

Page 24: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

24

Appendix C

Classrooms for the Future

Student Survey

Page 25: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Teacher Code: __________________________________ Subject (Select only one):

Literature Writing Other Language Arts Algebra Calculus General Math Geometry Trigonometry Other Math Chemistry Earth Science General Science Physics Other Science Civics History World Cultures Other Social Studies

Revised 2/5/07 5:29 PM

25

Page 26: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

26

Teacher Practice

Statement Strongly

Agree Agree Neutral DisagreeStrongly Disagree

1) My classes are helping me build skills like how to work independently, how to research online, how to work as part of a team, etc., that will help me be successful in the modern workplace.

2) In this class, how much of the time does the teacher spend on each of the following activities?

All or almost all

More than half

About half

Some, but less than half

Rarely or none of the

time a) Leading whole class discussions b) Lecturing or telling us about the subject c) Working with small groups of students d) Working with individual students e) Walking around the classroom observing and helping students

3) Different class activities require different types and amounts of thinking. How much time do you spend in this class, or in doing work assigned in this class, using the kinds of thinking listed below.

Almost all of the time

Quite a lot Some A little

None or almost none

a) Really complex thinking and problem-solving b) Learning processes and “steps” in processes and then following those steps on assignments or tests c) Learning information by listening or reading and then remembering that information on assignments or tests. d) Just kind of sitting back and not really concentrating on anything

4) Who controls or makes decisions about the following things in this class?

The teacher,

completely

The teacher, mostly

It’s about half and half

Students, mostly

Students, completely

a) The topics studied b) The way in which we study the topics c) Whether we work together or alone d) The specific topics of papers or assignments e) The criteria for grading assignments

Page 27: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

27

Impact on Student Activity

5) To what extent do you agree with the following statements? Strongly

Agree Agree Neutral Disagree Strongly Disagree

a) In this class, the topics just don’t interest me, so I do just enough to get by, and no more b) Some of the topics in my classes have been so interesting that I have done more reading or research outside of class to find out more c) This year, my classes have been more interesting than last year.

6) In this class, what percentage of time do students usually spend… More than

80% 61 to 80%

41 to 60%

21 to 40% 0 to 20%

a) Listening to the teacher as a whole class. b) Listening to other students discuss as a whole class c) Working by myself d) Working in groups e) Talking with the teacher “one to one” f) Working with the teacher in a small group g) Not really paying attention or doing what I am supposed to be doing h) Using technology

7) As you understand this class, about what amount of your grade is the result of the following kinds of work…

All or almost all

of the grade

More than

half of the

grade

About half of

the grade

Some, but less than half

of the grade

None or almost none

a) Class participation. b) Quizzes and tests c) Papers and reports d) Independent projects e) Oral reports or presentations f) Group projects g) Work that is simply handed in and graded by the teachers h) Work that is shared in a meaningful way with an audience outside the classroom

Page 28: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

28

Impact on Student Attitudes

8) To what extent do you agree with the following statements? Strongly

Agree Agree Neutral Disagree Strongly Disagree

a) In my classes, I was tested on my understanding and not just my memory. b) I am proud of my school. c) I get excited about going to school. d) I put more effort into school this year. e) I am able to use things I learn in one subject (for example mathematics) to better understand another subject (for example social studies) f) Compared to the beginning this year, I feel more confident about life after high school. g) I feel ready for the real world, with reference to my technology skills. h) The work I am doing in my classes will be useful to me in the job I hope to have as an adult i) I have often thought about becoming a teacher. j) I think teaching mathematics or science would be fun. k) I feel challenged at this school. l) I feel that if I work hard I can be successful in my classes m) In my classes, a lot of time is spent answering questions from a book or worksheet. n) The amount and rigor of class work my teacher assign is pretty typical of other classes I have taken. o) This school is providing me with a high quality education..

9) Rate your interest in each of the content areas below.

The most highly

interested Highly interested Interested A little interested Not at all interesteda) Reading/English language arts b) Math c) Science d) Social studies

Page 29: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

29

Factors that May Be Enhancing or Limiting Program Impact 10) How long does it take to:

1/2-1 hour

1 - 3 hours

1/2 - 1 full day

2 - 5 days

1 - 3 weeks

A month or more

a) Get a failed computer repaired? b) Get network services restored when the network goes down? c) Get help on a software problem or question? 11) Think about all of your English language arts, mathematics, social studies, and science classes. In these classes, how often do your teachers have you… Never

Rarely (a few times a year)

Sometimes (once or twice a

month) Often (once or twice a week) Almost Daily

a) use a word processor to write a story or report b) use software to learn and practice skills c) use a spreadsheet to enter and calculate numbers or create graphs for an assignment (Excel, etc) d) create a database of information for a class project (FileMaker Pro, Access, etc) e) create a presentation and present information to classmates or others (PowerPoint, etc) f) communicate by email with friends, experts, and others about topics you are studying g) Use online discussions to gather information for an assignment (discussion boards, videoconferencing, etc) h) conduct Internet research on an assignment topic i) use tools, such as graphing calculators or digital microscopes, to analyze information j) produce print products (with desktop publishing software) k) create multimedia reports or projects (with video, graphics, and sound editing) l) use technology to complete a test or quiz

Page 30: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Appendix D

Classrooms for the Future

Classroom Observation Tool

Revised 2/5/07 5:29 PM

30

Page 31: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

CLASSROOMS FOR THE FUTURE CLASSROOM DATA COLLECTION SHEET

Data Collector ID: ________________________________ Teacher ID: _____________________________ Date: _____/_____/_____ Period: _________________ Subject Observed: (Physics, Geometry, History, …): _____________________ Number of Students________ Number of Teachers, Teacher's Aides, etc.___________

Copyright, 2007

Page 32: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Learning Environment: 1) Access to technology (check all that apply):

Teacher has access to computer Presentation station is available (projector, speakers) Electronic Whiteboard is available 1 student per computer 2 students per computer 3-5 students per computer More than 5 students per computer Internet Access is available to all computers

2) Organization of Classroom (check one):

Traditional rows Small clusters of 3-5 student desks Science Lab Desks arranged in rows; students face each other Circles or semi-circles Computer lab Outside of classroom Classroom arranged like a lab, or with computers along the wall Other, please describe and illustrate below

Middle 1/3 of Class 32

Page 33: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Beginning 1/3 of the Class Period 3) Classroom Teacher Activity:

For each of the classroom activities below, estimate the percentage of time the teacher spent in each activity. The total should equal 100%.

_______ % Whole group lecture/instruction _______ % Leading a whole class discussion _______ % Working with a small group _______ % Working with individual students _______ % Walking, observing, and interacting with students _______ % Working at desk, or other professional work not involving students

4) Level of Engagement Scale: 1 = lightly engaged 5 = intensely engaged

Brief description of activity

% of Student Engaged

Level of Engagement

(1 to 5)

Beginning 1/3 of class

%

Instructional Practices:

5) Complexity 6) Instructional Style 7) Relevance Beginning 1/3 of the class

Basic Skills _ Higher Order

Didactic _ Constructivist

Artificial _ Real-World

Middle 1/3 of Class 33

Page 34: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Middle 1/3 of the Class Period 8) Classroom Teacher Activity:

For each of the classroom activities below, estimate the percentage of time the teacher spent in each activity. The total should equal 100%.

_______ % Whole group lecture/instruction _______ % Leading a whole class discussion _______ % Working with a small group _______ % Working with individual students _______ % Walking, observing, and interacting with students _______ % Working at desk, or other professional work not involving students

9) Level of Engagement Scale: 1 = lightly engaged 5 = intensely engaged

Brief description of activity

% of Student Engaged

Level of Engagement

(1 to 5)

Middle 1/3 of class

%

Instructional Practices:

10) Complexity 11) Instructional Style 12) Relevance Middle 1/3 of the class

Basic Skills _ Higher Order

Didactic _ Constructivist

Artificial _ Real-World

Middle 1/3 of Class 34

Page 35: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Final 1/3 of the Class Period 13) Classroom Teacher Activity:

For each of the classroom activities below, estimate the percentage of time the teacher spent in each activity. The total should equal 100%.

_______ % Whole group lecture/instruction _______ % Leading a whole class discussion _______ % Working with a small group _______ % Working with individual students _______ % Walking, observing, and interacting with students _______ % Working at desk, or other professional work not involving students

14) Level of Engagement Scale: 1 = lightly engaged 5 = intensely engaged

Brief description of activity

% of Student Engaged

Level of Engagement

(1 to 5)

Final 1/3 of class

%

Instructional Practices:

15) Complexity 16) Instructional Style 17) Relevance Final 1/3 of the class

Basic Skills _ Higher Order

Didactic _ Constructivist

Artificial _ Real-World

Classroom 1 FINAL 1/3 OF CLASS 35

Page 36: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Post Class Wrap-Up 18) Student Work/Products

Y N D/K N/A Are students working on a clearly defined task or tasks?

Y N D/K N/A Are students given the opportunity to demonstrate learning in a variety of ways (visual, auditory, kinesthetic)?

Y N D/K N/A Can students choose end products based on their own interests?

Y N D/K N/A Is there a clear rubric that will be used to assess project work?

Y N D/K N/A Are students working on different products?

Y N D/K N/A Can students choose to work on individual or group products?

Y N D/K N/A Are students allowed to work at their own pace?

*Choose N/A if students are not working on a product. Choose D/K if students are working on a project, but you do not know, or can’t tell about the specifics

Classroom 1 FINAL 1/3 OF CLASS 36

Page 37: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

19) Classroom Management:

Classroom management refers to procedures that are put into place to ensure efficient activity and minimize distractions. Based on your observation, which best describes the management of this classroom?

Excellent. Smooth flow from activity to activity. Procedures for dealing with access to teacher and resources clearly thought-out and adhered to. High levels of engagement. No problems or distractions.

Good. As above but with minor problems or distractions that are dealt with by the teacher.

Fair. Some disruption in flow of activities. Procedures for access are occasionally ignored or are unclear. Some evidence of disengagement. Problems or distractions, while not major, are not always dealt with efficiently.

Poor. No apparent flow from activity to activity. Procedures for access to teacher and resources either nonexistent or ineffective. High levels of disengagement. Problems and distractions are frequent and not effectively dealt with.

20) Teacher Comfort Level:

Could not tell

Teacher seemed completely uncomfortable

Teacher seemed fairly uncomfortable

Teacher seemed fairly comfortable

Teacher seemed completely comfortable

Classroom 1 FINAL 1/3 OF CLASS 37

Page 38: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

21) Hardware Use: Technologies Used By Teachers or Students (circle all that apply): Laptops T S Desktop

Computer

T S PDA’S T S

Calculators T S Cameras, Still or video

T S TV/VCR T S

Probeware T S Microscope camera/projector

T S LCD Projector

T S

22) Software Use: If the following general categories were used by teacher or students in the course of the lesson, circle “T” or “S”: Used

by…

Educational management software – e.g. attendance, grades, lesson plans: T

Drill & practice, ILS, or educational games S

Word processing T S

Data management (spreadsheets), graphing, or analysis software (EXCEL, SPSS, STATVIEW)

T S

Database software (FileMaker Pro, Microsoft Access) T S

Presentation software (PowerPoint, Keynote) T S

Email T S

Other communication tools (IM, discussion boards, video conferencing) T S

Desktop publishing software T S

Web publishing software T S

Internet for research T S

Multimedia reference CDs for research (e.g. online encyclopedias) T S

Simulations/modeling software T S

Software for video, graphics, and sound editing or production T S

Probeware (devices for gathering experimental data) T S

Web based digital curriculum or curriculum resources T S

Other Web-based learning activities T S

Other (describe): T S

Classroom 1 FINAL 1/3 OF CLASS 38

Page 39: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

23) Instructional Strategies – Specific For each of the instructional strategies below, indicate the extent to which that strategy was part of the lesson you observed. Lessons may contain more than one strategy. Consider each strategy individually.

Extent to which this strategy was used during the lesson

Instructional Strategy None Little Fair

amount Most All

Teacher lecture or demonstration Teacher-led discussion – low level, factual

Teacher-led high-level discussion w/ purposeful questions to students

Project or problem based learning

Authentic learning Multi-modal teaching (ie, visual, auditory, kinesthetic)

Peer teaching Collaborative learning – informal Collaborative learning – w/ formal roles WebQuests Learning centers

24) New Tools For each of the tools listed below, indicate which, if any, were used during the lesson.

Extent to which this strategy was used during the lesson Instructional

Strategy Not Used

Teacher-made

Student-made

Made by

others DK

Websites Podcasts Blogs Wikis Digital Photos and/or Movies

Classroom 1 FINAL 1/3 OF CLASS 39

Page 40: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

25) Technology Usage Summary: Based on this observation, estimate the percentage of students who actively used technology:

_______ % Not at all _______ % Briefly during the period _______ % About one fourth of the period _______ % About half of the period _______ % About three fourths of the period _______ % Almost the entire period 26) Percentages of Student Activity: Based on this observation, estimate the percentage of time students were engaged in the following:

_______ % Listening to the teacher (in a large group setting) _______ % Listening to other students (in a large group setting) _______ % Working independently _______ % Working in groups _______ % Talking with the teacher in 1-to-1 or small group

conversations _______ % Off Task (not doing what the teacher intended) 27) 21st Century Skills, Proficiencies, or Dispositions:

To what extent did this lesson make an effort to SPECIFICALLY PROVIDE INSTRUCTION IN, OR EMPLOY STRATEGIES TO DEVELOP the following skills:

1 = Not at all 2 = Somewhat 3 = Substantially NA = Not applicable/can’t tell Visual Literacy 1 2 3 NA Scientific Literacy 1 2 3 NA Cultural Literacy OR Global Awareness 1 2 3 NA Teaming OR Collaboration Skills 1 2 3 NA E-Communication Skills 1 2 3 NA Social OR Personal Responsibility 1 2 3 NA Self-Direction 1 2 3 NA Creativity 1 2 3 NA Higher-Order Thinking 1 2 3 NA Use of Real World Tools 1 2 3 NA Ability to Produce High-Quality Products 1 2 3 NA Planning, Prioritizing, and Managing Work 1 2 3 NA

Classroom 1 FINAL 1/3 OF CLASS 40

Page 41: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Appendix E

Classrooms for the Future

Pennsylvania Technology Inventory Survey Items to be used in the CFF Evaluation

Revised 2/5/07 5:29 PM

41

Page 42: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

PATI Survey Items to be used in CFF Evaluation (CFF-relevant items extracted from the Pennsylvania Technology Inventory, and annual survey of all teachers.) 11. How much time per week does a typical student in your class(es) use technology while at school? (Select one) - Not at all - Less than 30 minutes per week - 30 to 60 minutes per week - 1 to 2 hours per week - More than 2 hours per week 12. How has the amount of time that students use technology in your classroom changed since last year? - Students spend significantly less time - Students spend a little less time - Students spend about the same time - Students spend a little more time - Students spend significantly more time 15. How often do students in your class(es) use technology to do the following? **Mark "Not Applicable" only if this use does not apply to your subject area: * Communicate (for example, over email or through discussion boards) with experts, peers, and others - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable * Solve real-world problems (i.e. involving situations, issues, and tasks that people actually tackle in the outside world) - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable

Revised 2/5/07 5:29 PM

42

Page 43: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

* Produce print products - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable * Produce multi-media, Web, or presentation products - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable * Conduct online research - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable * Use drill and practice or tutorial software - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable * Use the Internet to collaborate with students in or beyond your school - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable

Revised 2/5/07 5:29 PM

43

Page 44: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

* Visually represent or investigate concepts (e.g., through, concept mapping, graphing, reading charts) - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable * Use digital tools and peripheral devices (e.g., digital cameras, probes, scanners) to enhance their learning or their school work - Daily - Weekly - Monthly - Quarterly - Rarely or Never - Not Applicable 19. Which technologies do you require students to use for your classes? (Check all that apply) - Word processing/document processing - Spreadsheets (for data analysis and management) - Other data analysis (SPSS, Fathom, Mathmatica) or database software (Microsoft Access, Filemaker Pro) - Email (including attachment and address book features) and web browsers (including book-marking, "back" or "home" features) - Presentation software (PowerPoint, Astound, etc.) - Multimedia editing or authoring tools like Authorware, Hyperstudio Photoshop, Illustrator, or video editing technology - Graphic peripherals (for example, scanners, digital cameras, etc.) - Electronic information sources like the WEB, ERIC, EBSCO (searching these efficiently, for example by using and/or to narrow/expand your search; identifying synonyms or keywords) - Technologies specific to your field (for example, probeware in the sciences, geographic information systems in the social sciences, etc.) 24. How prepared do you feel to manage technology-supported learning with your class(es)? (Select one) - I have a variety of classroom management and organizational strategies for using technology. I know I can smoothly orchestrate technology-supported learning activities in a variety of settings and ways (whole class, small group, centers in labs or the classroom). - I have some classroom management and organizational strategies but think I need more. - I have very few classroom management and organizational strategies for using technology.

Revised 2/5/07 5:29 PM

44

Page 45: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

28. Which best describes your skill level with each of the following technologies: * Word processing/document processing - No Skill - Novice - Intermediate - Highly skilled * Spreadsheets (for data analysis and management) - No Skill - Novice - Intermediate - Highly skilled * Other data analysis (SPSS, Fathom, Mathematica) or database software (Microsoft Access, Filemaker Pro). - No Skill - Novice - Intermediate - Highly skilled * Email (including attachment and address book features) and web browsers (including book-marking, "back" or "home" features) - No Skill - Novice - Intermediate - Highly skilled * Presentation software (PowerPoint, Astound, etc.) - No Skill - Novice - Intermediate - Highly skilled * Multimedia editing or authoring tools like Authorware, Hyperstudio Photoshop, Illustrator, or video editing technology - No Skill - Novice - Intermediate - Highly skilled * Graphic peripherals (for example, scanners, digital cameras, etc.) - No Skill - Novice - Intermediate - Highly skilled

Revised 2/5/07 5:29 PM

45

Page 46: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

* Electronic information sources like the WEB, ERIC, EBSCO (searching these efficiently, for example by using "and"/"or" to narrow/expand your search; identifying synonyms or keywords) - No Skill - Novice - Intermediate - Highly skilled * Technologies specific to your field (for example, probeware in the sciences, geographic information systems in the social sciences, etc.) - No Skill - Novice - Intermediate - Highly skilled 29 (Part C): I am comfortable planning for class sessions that involve students using technology during instruction. - Strongly Agree - Agree - Disagree - Strongly Disagree 35 Rate your access to the following items while at school: (Part D): * Reliability of computers, printers, projectors, and other equipment (i.e. it works when I need it) - Non-existent - Very Poor/ Barely adequate - Adequate or Pretty good - Good - Excellent (Part E): * Reliable, high-speed access to the Internet in the classrooms, labs, and media centers. - Non-existent - Very Poor/ Barely adequate - Adequate or Pretty good - Good - Excellent

Revised 2/5/07 5:29 PM

46

Page 47: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

(Part I): * Technical support with little or no wait-time - Non-existent - Very Poor/ Barely adequate - Adequate or Pretty good - Good - Excellent (Part J): * Instructional support that helps me to integrate technology - Non-existent - Very Poor/ Barely adequate - Adequate or Pretty good - Good - Excellent 49. During this school year: * My students have had the opportunity to work on projects or assignments that involve collaborating with organizations (environmental groups, businesses) or individuals in their community - Yes - No - Don't Know * Students in other classes in this school have had the opportunity to work on projects or assignments that involve collaborating with organizations (environmental groups, businesses) or individuals in their community. - Yes - No - Don't Know

Revised 2/5/07 5:29 PM

47

Page 48: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

Revised 2/5/07 5:29 PM

48

Appendix F

The Teaching Performance Record (TPR)

Observation Tool

Page 49: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

49

Revised 2/5/07 5:29 PM

Page 50: Classrooms for the Future Evaluation Plan for Year One · "Classrooms for the Future" Evaluation Plan for Year One . ... Pennsylvania Governor Ed Rendell announced that the CFF initiative,

50

Revised 2/5/07 5:29 PM