Sid the Science Kid Summative Evaluation · G O O D M A N R E S E A R C H G R O U P , I N C . M a r...

G O O D M A N R E S E A R C H G R O U P , I N C . A u g u s t 2 0 0 3 1

Sid the Science Kid Summative Evaluation

PREPARED BY Elizabeth R. Bachrach, Ph.D. Markeisha Grant, B.A. Irene F. Goodman, Ed.D.

SUBMITTED TO The Jim Henson Company Thirteen/WNET March 2012

TABLE OF CONTENTS

Executive Summary ......................................................................................... i Introduction ..................................................................................................... 1

Sid the Science Kid Science Camp Goals .................................................. 1

Evaluation Research Questions .................................................................. 2

Methods .......................................................................................................... 2

Research Design ......................................................................................... 2

Data Collection: Teachers and Students ..................................................... 3

Profile of Study Participants ....................................................................... 5

Research Context: Science in the Classrooms ............................................ 6

Results ........................................................................................................... 10

Short Term Effects on Children ............................................................... 10

Teachers’ Knowledge and Comfort: Changes Seen over time ................. 23 Treatment Group Teachers’ Satisfaction .................................................. 27

Conclusions ................................................................................................... 30 Recommendations ......................................................................................... 31

G O O D M A N R E S E A R C H G R O U P , I N C . M a r c h 2 0 1 2 i

EXECUTIVE SUMMARY Sid the Science Kid (STSK) is a daily PBS series, with accompanying website and outreach materials, produced by The Jim Henson Company in Los Angeles. The program aims to engage three- to six-year-old children and their adult caregivers. Goals of the series are to:

� build on preschoolers’ natural curiosity about the world to encourage children to think, talk, and work the way scientists do;

� partner with parents and teachers to create a climate of curiosity for children; and

� contribute to school readiness by fostering children’s skills, motivation, and confidence as learners.

KCET Education, Children’s Programming, and Outreach teams created the Sid the Science Kid Science Camp program for use in early childhood education settings. Production and distribution is now being conducted by Thirteen/WNET Children’s Outreach. Similar to the TV series, the Science Camp curriculum lessons focus on one theme per week and the camp goals match those of the series. An initial pilot Science Camp curriculum was built around the Science Tools episode cycle. Jim Henson Company (JHC) contracted with Goodman Research Group, Inc. (GRG), an evaluation research firm in Cambridge, MA to conduct summative evaluation of the Science Camp’s week-long pilot curriculum. The evaluation was designed to assess short-term effectiveness of the new curriculum for students and teachers, and to obtain feedback from teachers about the curriculum and accompanying training. Evaluation research questions were:

1. How effective was the curriculum in teaching preschool aged children the science content presented through the week-long unit?

2. What short-term student outcomes are evident after the week-long curriculum?

3. What were teachers’ perceptions of the curriculum including ease of use, their own learning, and their students’ learning? How prepared did they feel to use the curriculum in their classrooms after the training?

GRG designed and employed a quasi-experimental two-group, pre-post design for the evaluation. Early childhood teachers in two cities (New York City, NY and Los Angeles area, CA) were recruited to participate with their classroom students. In all, 16 teachers and 220 students participated; half were assigned to the Treatment group and the other half to the Waiting Control group. At two points in time -- once before and once after the week-long Science Camp curriculum was implemented in Treatment group classrooms -- teachers completed surveys and students participated in one-on-one play activities and interviews with field interviewers hired and trained by GRG.

G O O D M A N R E S E A R C H G R O U P , I N C . M a r c h 2 0 1 2 ii

STUDY PARTICIPANTS AND RESEARCH CONTEXT Participating students were a fairly equal number of boys and girls, mostly four years old. About 50% of the students were English language learners (ELL) and the majority spoke a language other than English at home, primarily Spanish. All participating teachers were women, with an average of 10 years teaching experience. Most were Hispanic or Latina, and held a Bachelor’s or Associate’s degree. All of the teachers were accustomed to leading students in hands-on science activities, making use of small- and large-group work. During the intervention week, the primary difference between the Treatment and Control group classrooms was the specific content of the Science Camp curriculum and the use of Sid the Science Kid video as a medium through which to engage students and demonstrate the science content (rather than the teachers alone presenting the content). At the two testing points, interviewers worked with students individually for about 15 minutes per session. In an area separate from the rest of the students, interviewers laid out a standard set of materials on a table and invited each child to select and play with any of the objects, including: magnifying glasses, Popsicle sticks, notebooks/journals, rulers, charts, stickers, and crayons. During the session, interviewers recorded the child’s spontaneous activity and verbal comments, and asked probing questions about their actions. Following a protocol developed by GRG, the interviewers listened specifically for use of relevant vocabulary and evidence of the students’ curiosity and exploration; they recorded children’s behavior under one of four activities that corresponded with those included in the Science Camp curriculum: Magnification, Measurement, Charts, and Science Journals. The same protocol was used for both Treatment and Control groups. KEY STUDENT FINDINGS After one week of Sid Science Camp activities and accompanying video, children in the Treatment group were able to make real-world connections with the activities they conducted in their classes. They showed interest in the activities they had done, and excitement about continuing to try out those activities on their own time. Their spontaneous use of words related to specific science tools, particularly those related to measurement and magnification, more than doubled. Analyses revealed a statistically significant positive shift in Treatment group children’s awareness and understanding of science and science tools compared to children in the Control group. This shift was demonstrated by the following among Treatment group children only:

→ Verbalized and demonstrated more sophisticated understanding of science, scientists, and science tools

→ Spontaneously used more science tool-specific words during their play with the objects

→ Spent more time using the interviewer-provided objects as science tools and in ways similar to the Science Camp classroom activities

G O O D M A N R E S E A R C H G R O U P , I N C . M a r c h 2 0 1 2 iii

→ Exhibited increased awareness of the function and purpose of specific science tools in their play

An association between these changes and the Science Camp curriculum, including the activities and video clips, was evident. After exposure to the program, children in the Treatment group:

→ Referred to STSK video clips during free play in and outside of the classroom and during play activities with interviewers

→ Replicated activities they saw Sid and his friends doing in the video clips → Referred to activities they had conducted in class and to specific terms

their teachers taught them Treatment group children’s behaviors and comments made during the play activities suggested they were processing and working toward a deeper understanding of the science concepts they had learned over the past week. KEY TEACHER FINDINGS Overall, Treatment group teachers were very satisfied with the Science Camp curriculum including the one-day training session, the ease of implementation, and the ease with which the activities fit into their regular daily schedules. Compared to teachers in the Control group, those who conducted the Science Camp activities with their students perceived personal knowledge gains and increased confidence with science content. They also showed increased comfort with science teaching strategies and reported that their students showed more interest in science learning as a result of the Science Camp activities. KEY RECOMMENDATIONS Highlight for teachers the opportunity to take more time to implement the curriculum in class.

� A few teachers mentioned that a week was too short for the implementation of the curriculum; they would like to dedicate two or more days to each activity to ensure that students grasp the concept of using different scientific tools.

� Emphasize these options, which are already included in the teachers’ manual, for extending each lesson with additional activities.

� Considering the built-in flexibility to expand the activities over time, teachers can tailor the curriculum to fit their students’ needs.

� Extended use of the curriculum may increase the likelihood of longer-term positive effects on the students.

Spend more time on -- and include developmentally appropriate activities specifically dedicated to -- the use of science journals to record children’s own science investigations and observations.

� In their one-on-one play activities, children did not use the science journals as much as they did other science tools provided. Teachers had reported that the journal activity was the least successful because students did not understand the idea of recording their own observations.

G O O D M A N R E S E A R C H G R O U P , I N C . M a r c h 2 0 1 2 iv

� Specific activities focused solely on this science tool may help children to understand the concept.

Incorporate more repeated use of and emphasis on general science terms as well as those specific to each lesson.

� The words that children heard many times and that had specific activities associated with them (e.g., “magnify,” “look bigger,” “measure”) were the words that they incorporated into their vocabularies.

� In order for students to learn more general science words such as “compare,” “explore,” and “observe,” those words should be more prevalent throughout the lessons.

Evaluation findings can be interpreted as an illustration of the value added from presenting science concepts and activities in the context of a lively and engaging television series that features appealing characters that children enjoy, relate to, and learn from (GRG, 2008), along with a high level of energy and music. As it is further developed, with additional lessons added, there is potential for a very successful early childhood science program.

G O O D M A N R E S E A R C H G R O U P , I N C . M a r c h 2 0 1 2

1

INTRODUCTION Sid the Science Kid (STSK) is a daily PBS series produced by The Jim Henson Company in Los Angeles. Via a daily television series, website, and outreach initiative, STSK aims to engage three- to six-year-old children and their adult caregivers. In summer 2010, the KCET Education, Children’s Programming, and Outreach team created the Sid the Science Kid Science Camp program for use in early childhood education programs, beginning with one week-long pilot curriculum. Preliminary testing in two Head Start locations in Los Angeles in August 2010 led to modifications in preparation for national summative evaluation of the pilot week. In January, 2011, Thirteen/WNET Children’s Outreach, in New York City, took over the development and distribution of the Sid Science Camp curriculum. The Jim Henson Company (JHC) contracted with Goodman Research Group, Inc. (GRG) to conduct summative evaluation of the Science Camp’s week-long pilot curriculum. GRG, an evaluation research firm in Cambridge, MA that specializes in the evaluation of educational programs, materials, and services, had previously conducted a national summative evaluation of the STSK television series, website, and outreach in its the first season (Fall 2008). SID THE SCIENCE KID SCIENCE CAMP GOALS The Science Camp curriculum was developed with the same overall goals as the series:

� build on preschoolers’ natural curiosity about the world to encourage children to think, talk, and work the way scientists do;

� partner with parents and teachers to create a climate of curiosity for children; and

� contribute to school readiness by fostering children’s skills, motivation, and confidence as learners.

The television series was modeled after the Preschool Pathways to Science (PrePS) curriculum developed by Rochel Gelman, Ph.D. and Kimberly Brenneman, Ph.D. at Rutgers University and Gay E. MacDonald and Moises Roman at UCLA. Each week of STSK focuses on one theme (e.g., science tools) that is addressed through all five episodes for that week. The initial week-long Science Camp curriculum was built around the Science Tools Episode Cycle and integrates the following educational philosophy:

� Preschoolers are naturally curious. � Early exposure to science can inspire a positive lifelong attitude toward it

by empowering children to see themselves as capable learners, and motivating them to learn and do more.

� Learning to think and act like a scientist is more important than memorizing discrete facts.


2

Beginning in summer 2011, GRG conducted summative evaluation of the week-long pilot Sid the Science Kid Science Camp curriculum in two national sites: New York City, NY and the Greater Los Angeles area, CA. EVALUATION RESEARCH QUESTIONS The evaluation was designed to assess the short-term effectiveness of the new curriculum using the following research questions:

1. How effective was the curriculum in teaching preschool aged children the science content presented through the week-long unit?

2. What short-term student outcomes are evident after the week-long curriculum?

3. What were teachers’ perceptions of the curriculum including ease of use, their own learning, and their students’ learning? How prepared did they feel to use the curriculum in their classrooms after the training?

METHODS RESEARCH DESIGN GRG designed and employed a quasi-experimental two-group, pre-post design for the evaluation. Early childhood teachers in two cities (New York City and Los Angeles area) were recruited to participate with their classroom students. GRG obtained parental consent for all participating students. GRG randomly selected 8 of the 16 teachers, and their students, to be in the Treatment group and the other 8 to be in the Waiting Control group. All 16 teachers and students participated in GRG evaluation activities, described below. The Treatment group teachers attended a one-day in-person training at either Thirteen/WNET (in New York) or Jim Henson Studios (in Los Angeles) and then conducted the week-long Science Camp activities (including the accompanying videos and other materials) with their students. The Waiting Control group neither received training nor conducted Science Camp activities. Teachers in the Waiting Control group (referred to from this point forward as Control group teachers) were invited to receive all Science Camp materials and training after completion of the data collection for the evaluation study. Each teacher received a stipend for their full participation.


3

DATA COLLECTION: TEACHERS AND STUDENTS Teacher Surveys All teachers completed a paper survey before and after the week-long intervention. For the Treatment group, surveys were completed before and after use of the Science Camp curriculum. Teachers in the Control group completed surveys during the same time period, before and after a week of business as usual in their classrooms. The pre-surveys included questions about the teachers’ classroom characteristics, regular science procedures, and their own experiences and comfort teaching science. Post surveys included questions about activities conducted over the previous week (i.e., during implementation of the Science Camp for Treatment group; during a regular week for the Control group), as well as questions about experiences and comfort with science, for comparison to their pre-survey responses. Additionally, teachers in the Treatment group commented on their use of and satisfaction with the Science Camp curriculum. Student Play Activities Data were collected from students on two separate occasions during this evaluation study: once before (Time 1) and once after (Time 2) the week-long intervention period. Field researchers (hereafter referred to as “interviewers”), hired and trained by GRG, visited the classrooms, and conducted one-on-one play activities and brief interviews with each student; sessions lasted from 10 to 15 minutes per child. Using a standardized package of materials provided by GRG, interviewers recorded their observations of students’ actions and behaviors on a paper observation protocol and later entered their notes in a secure online form programmed by GRG. Interviewers were blind to the experimental condition the children were assigned to, and knew very little about the intervention itself. GRG’s aim was for the interviewers to record children’s behavior with as few biases or expectations as possible. Each interviewer package contained materials that were similar to those used in the Science Camp activities and were likely to be familiar to students as common classroom objects. Each package consisted of the following:

� Magnifying glasses � Charts � Popsicle sticks � Stickers � Notebooks/Journals � Crayons � Rulers

At each site, interviewers conducted the play activities with children individually, in an area of the classroom or school selected by the teachers. Interviewers laid out the materials on a table and invited each child to select and play with any of the objects. During the session, interviewers recorded the child’s spontaneous


4

activity and verbal comments, and asked probing questions about their actions. The interviewers listened specifically for use of relevant vocabulary and evidence of the students’ curiosity and exploration. According to the observation protocol and GRG training, interviewers recorded the children’s behavior under one of four activities that corresponded with those included in the Science Camp curriculum: magnification, measurement, charts, and science journals. Examples of children’s actions with the materials and the activities under which interviewers recorded them follow:

Because the interviewers did not know about the Science Camp curriculum, or the child’s experimental condition, GRG researchers relied on the detailed information interviewers provided as they described each child’s activities. The protocol included space for interviewers to indicate which materials the children selected, the activities they engaged in with those materials, the number and length of discrete activities (i.e., the length of time children spent with one or more objects until they put them down and selected a new object), and the presence or absence of several science-related terms and phrases. Classroom Observations GRG researchers visited each location once during the week of the Science Camp intervention to observe the lessons as they were implemented. Using an observation protocol, researchers recorded a snapshot of the pilot curriculum in use and noted teachers’ and students’ behaviors and response to the activities. See Appendix A for copies of all data collection instruments; data from teachers’ surveys are included.

Look at the objects around them with magnifying glass.

Magnification

Compare length of the ruler to the length of the notebook.

Measurement

Draw and describe work in the notebook.

Journal

Place stickers in rows and columns on the chart paper.

Charts


5

PROFILE OF STUDY PARTICIPANTS In New York, participants were teachers in eight Head Start classrooms and their students (N=124 students). In Los Angeles, participants were teachers in two Los Angeles Universal Preschool (LAUP) classrooms. Each LAUP classroom had one head teacher and three teaching assistants, and had a morning session and an afternoon session with 24 students in each (N=96 students). All 16 teachers were women. Shown in Table 1, most were Hispanic or Latina, and held a Bachelor’s or Associate’s degree. On average, teachers had 10 years of teaching experience. Their participating students were a fairly equal number of boys and girls, mostly four years old. About 50% of the students were English language learners (ELL) and the majority spoke a language other than English at home, primarily Spanish. Table 1 Participant Demographics Number of

Teachers (N=16)

Race/Ethnicity Latino/a or Hispanic 11 White 2 Black or African-American 2 Prefer not to respond 1 American Indian or Alaskan Native – Asian – Native Hawaiian or Other Pacific Islander –

Education Some high school 1 High school diploma or GED - Associate’s (2 year) college degree 5 Bachelor’s (4 year) college degree 9 Master’s degree 1 Doctorate/other post graduate -

% of children (N=220)

Gender Girls 46% Boys 54%

Age 3 years old 10% 4 years old 66% 5 years old 22%


6

RESEARCH CONTEXT: SCIENCE IN THE CLASSROOMS The following section describes participants’ typical classroom arrangements and science teaching, as well as the specific science activities that were conducted during the week of the Science Camp intervention. Center and Classroom Characteristics In the New York Head Start program, each classroom was created from one very large space divided into two separate rooms. The divider was a unit of shelves; teachers could see over the shelves, and students could not. Each individual classroom was quite spacious and allowed for children to move freely around the room during their free play time. GRG researchers did not observe any outdoor activity. In Los Angeles, classrooms were smaller, and were in bungalow-style buildings. Each classroom doorway led outside, rather than to a shared indoor hallway, and there was a door between the two classrooms. There was a large outdoor area with tricycle paths and several tricycles, a climbing structure, swings, and other play equipment. The researcher observed outdoor playtime in addition to the Science Camp activity. Classrooms in both locations were brightly lit and were arranged with tables and different “centers” that are common in preschool classrooms. Children appeared comfortable moving from one area to another during their free play times. In five of the 10 classrooms, head teachers had between one and three classroom assistants. Recall, there were eight separate classrooms in New York, each with one head teacher, and there were two separate classrooms, with 4 teachers in each, in Los Angeles. Among the five Treatment group classrooms, three had access to DVD Players and TVs, and two had access to personal computers with an Internet connection, printers, and SMART Boards. Typical Approach to Science Teaching Aside from the one-day Sid Science Camp training attended by the Treatment group teachers, the majority of participating teachers had not attended any previous training on how to conduct science activities in the classroom. Shown in Table 2, teachers in both groups were accustomed to conducting hands-on science activities with their students in small and large groups.

� 14 teachers had a designated science area in the classroom; 6 of them also had a designated area outside

� 15 had children working in small groups with some doing science and some doing a different activity

� 10 gave children the choice to do a science activity or a different activity � 9 typically offered science activities every day

All participating teachers were accustomed to leading students in hands-on science activities, making use of small- and large-group work.


7

Table 2 Teachers’ Approach to Science # of

Teachers Small group activities 14 Kids do almost all hands- on activities 11 Large group activities 10 Kids do some hands-on exploration 6 I lead science activities; kids watch as I handle the materials 3 Kids do very little hands-on exploration 1 N=16 All teachers, in both groups, used as the core curriculum in their classroom The Creative Curriculum, which operates under the philosophy that children learn best by actively thinking and experimenting on their own to find out how things work. The curriculum focuses on social and emotional development, the arts, physical health and development, mathematics and science. The lessons involve teacher-led, small-group, and large-group activities. Similar to the educational philosophy of the Sid Science Camp, the goal of The Creative Curriculum is to encourage students to become active learners who try out their own ideas and develop their own opinions on topics. Teachers’ Use of Science Tools during the Intervention Week During the week, teachers in the Treatment group used the science tools as suggested in the Science Camp manual. More than half of the Control group teachers also conducted activities that involved magnifiers and linear and standard measurement. Half or fewer of the Control group teachers also used data and charts, journals, and non-standard measurement activities. See Table 3. Table 3 Teachers’ Use of Activities Involving Science Tools # of Teachers Treatment Control Use magnifiers in science explorations. 8 7 Activities about linear measurement. 7 5 Activities about standard measurement. 7 5 Collect data and record it on a chart.* 8 4 Record observations in journals.* 8 3 Activities about non-standard measurement.* 8 3 Discussions about linear measurement.* 7 2

N=16 * Done by more Treatment group teachers than Control group teachers Commenting at Time 2 about the success of activities they conducted during the previous week, most of the Treatment group teachers considered the activities

During the intervention week, teachers in both groups reported conducting at least some classroom activities that made use of science tools. Treatment group teachers conducted more than did Control group teachers.


8

with magnifiers, collecting and recording data, and non-standard measurement to be the most successful. Students “loved using the magnifiers, rulers, and creating charts,” and some teachers have continued “making charts to gather many kinds of information.” One teacher explained that students enjoyed the activities around non-standard measurement, “especially when they used their shoes, hands, body, blocks etc.” Three of the Control group teachers said the activities they conducted with magnifiers were the most successful. The remainder of Control group teachers described science-related activities distinct from the Science Camp curriculum, such as mixing colors, growing seeds, and growing crystals (children viewed the crystals with magnifying glasses) as the most successful over the past week. Six Treatment group teachers considered the time spent recording observations in journal in the previous week to be the least successful. Many felt it was not age- appropriate for preschool students who had difficulty understanding the concept of recording information in this way. One teacher explained, “The children found it harder to record their thoughts with pictures than to verbalize what they had observed or explored.” A different teacher noted that size was a barrier: “The journals were too small for them to draw. The children are used to drawing on bigger paper.” During the on-site observations, GRG researchers noted that Treatment group children showed difficulty generalizing from what they saw on TV (i.e., Sid and his friends drew pictures of the rolie polies they viewed with the magnifiers) to what they saw in themselves (i.e., in class, they viewed rocks, shells, and leaves with magnifiers). Instead of recording what they saw in class, the children drew what they saw drawn on TV. Table 4 Most and Least Successful Activities in the Past Week # of Teachers Most

Successful Least

Successful Use magnifiers in science explorations. 9 0 Activities about non-standard measurement. 9 1 Collect data and record it on a chart. 7 1 Activities about standard-measurement. 5 0 Record observations in journals. 3 6 Activities about linear measurement. 2 3 Discussions about linear measurement. 2 2 Growing seeds 2 0 Rock candy/growing crystals 2 0 Farms 1 1 Mixing colors 1 0 Fossils 0 3

N=16


9

Treatment Group Teachers’ Use of Video Clips Treatment group teachers viewed both the STSK program segments (animated) and the Fab Lab segments (not animated) that were provided with the curriculum. They described the two types of video segments as “both good,” with no preference for one type over the other. One teacher showed students additional video segments as well (e.g., segments from the Internet). Most teachers introduced the video segment, “explained any vocabulary that was not familiar for the children,” viewed the segment, and then reviewed the content and implemented the activities. Two of them paused the videos at points, to repeat questions posed, or to “have the children predict. Then we continued the video.” Overall, during the intervention week, the primary difference between the Treatment and Control group classrooms was the specific content of the Science Camp curriculum and the use of Sid the Science Kid video as a medium through which to engage students and demonstrate the science content rather than the teachers alone presenting the content. Findings presented in this report can be interpreted as an illustration of the value added from presenting science concepts and activities in the context of a lively and engaging television series that features appealing characters that children enjoy, relate to, and learn from (GRG, 2008), along with a high level of energy and music.

Treatment group teachers incorporated STSK video segments throughout the week’s activities, as prescribed in the teacher’s manual.


10

RESULTS In this section, we present a summary of findings regarding the effects of the Science Camp curriculum on students. This is followed by teachers’ perceptions of the effects on students and themselves, changes seen over time in teachers across both groups, and finally Treatment group teachers’ satisfaction with Science Camp materials and training and the experience overall. SHORT TERM EFFECTS ON CHILDREN Overall, after one week of exposure to the Sid Science Camp activities and accompanying video, children in the Treatment group revealed a significant positive shift in their awareness and understanding of science and science tools, compared to children in the Control group. This shift was demonstrated by the following among Treatment group children only:

→ Verbalized and demonstrated more sophisticated understanding of science, scientists, and science tools

→ Spontaneously used more science tool-specific words during their play with the objects

→ Spent more time using the interviewer-provided objects as science tools and in ways similar to the Science Camp classroom activities

→ Exhibited increased awareness of the function and purpose of specific science tools in their play

The behaviors listed above were demonstrated by one third of Treatment group children before the week and by over one half of them after the Sid Science Camp week. Moreover, the majority (up to 87%) of children in the Treatment group used objects deliberately and accurately as science tools after the week, compared to half of the Control group children. An association between these changes and the Science Camp curriculum, including the activities and video clips, was evident. At Time 2, children in the Treatment group:

→ Referred to STSK video clips during free play in and outside of the classroom and during play activities with interviewers

→ Replicated activities they saw Sid and his friends doing in the video clips → Referred to activities they had conducted in class and to specific terms

their teachers taught them Interviewers’ descriptions of children’s behaviors and comments made during the play activities, suggested that children in the Treatment group were processing and working toward a deeper understanding of the science concepts they had learned over the past week. Summarized in Table 5, and described in more detail in sections that follow, some changes in behavior over time that reflected science-related thinking were


11

seen in both groups of children while other, more Science Camp-specific changes, were seen among the Treatment group children only (that is, no changes were evident among Control group children). For example, while all children used more of the objects provided at Time 2, the children in the Treatment group used them more deliberately as science tools. Table 5 Changes in Behavior from Time 1 to Time 2

Evident among Children in Both Groups

Treatment Group Outperformed Control Group

Children used more of the various objects provided by the interviewer.

Children engaged for longer with each science-tool related activity.

Children used the objects provided in ways that were appropriate to the tools’ purpose.

Children used the tools in ways that reflected learning from the Science Camp activities.

Few children used general science terms including “compare,” “contrast,” “observe,” “science.”

Children’s spontaneous use of science tool-specific terms more than doubled.

About two-thirds of children recalled having done similar activities before (e.g., at Time 1, in their classroom, at home).

Children referred specifically to Science Camp activities they had conducted in class, learned about from their teacher, and saw on TV (video clips).

Use of Science-Related Terms The most notable effects of the Sid Science Camp curriculum were revealed in children’s spontaneous use of terms specific to each science tool. Throughout the one-on-one play activities, interviewers recorded children’s comments and descriptions of their actions with the materials provided. Within each activity, interviewers noted whether or not children used general science-related terms such as “science,” “scientist,” “compare,” and “explore,” as well as terms related to each specific science tool. The key words specific to each of the four Sid Science Camp activities included the following. CHART: MEASUREMENT :

� “Information” � “Ruler” � “Chart” � “Non-standard” � “Data” � “Measure”

MAGNIFYING GLASS SCIENCE JOURNAL

� “Magnify” � “Journal” � “Tiny” � “Draw” � “Look Bigger” � “Record”

Combined across the four activities, children in the Treatment group showed a significant increase in the number of science tool-specific terms they used at Time 2. These terms appeared to be new to the children. Very few children in either group used any of the words during the play activity at Time 1. Children in

After participating in the Sid Science Camp activities for a week, children’s use of words related to specific science tools, particularly those related to measurement and magnification, more than doubled.


12

the Treatment group showed a 70% increase in use of all science tool-related terms at the end of the week, at Time 2. In contrast, there was no change among children in the Control group. See Figure 1 for average number of terms used by children in both groups, at Time 1 and Time 2. Figure 1 Children’s Spontaneous Use of Science Tool Terms during Play Activities

N=210 Compared to the increases seen in children’s use of terms specific to each science tool, over time, fewer children spontaneously used general science terms such as “compare,” “contrast,” “observe,” “science,” or “scientist.” Understanding of Science Tools and Their Functions As described above, during each one-on-one play activity (i.e., once before and once after the intervention week), the interviewer presented the child with seven objects and encouraged the child to select and play with any of those objects. During the 10-15 minute session, the interviewer noted which objects the child selected and recorded the child’s verbal and non-verbal actions. During their second play activity with interviewers (i.e., at Time 2), about two thirds of the children in both groups recalled having done similar activities in the past, and about one third said they had not. About 10% of the children in the Treatment group, compared to 2% of the Control group, referred to a specific activity they had done in class during the past week. For example:

“Um, yes, like non-standard measurement--you just get something like this and decide how big it is with a tool.” “We played with the magnifying glass and saw chart data.”

0.47

0.80

0.47

0.38

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Time 1 Time 2

Av

ere

ag

e N

um

be

r o

f W

rod

s U

sed

Treatment Control

At Time 2, children in the Treatment group showed more understanding of the objects provided as “science tools” compared to children in the Control group.

At Time 1, one third of Treatment group children used at least one of the science tool terms, and at Time 2, more than half of them did. There was no change for the Control group.


13

In the following sections, we describe children’s use of the science tools as they corresponded with the four main activities: Magnification, Charts, Measurement, and Science Journals. Children’s interactions with the objects were often similar across the whole sample at Time 1, with changes evident in the Treatment group at Time 2. For each activity, a summary of overall use is followed by group differences in behaviors (between the Treatment and Control groups), and finally examples of changes from Time 1 to Time 2 among the Treatment group only. Magnification

The magnifying glass was used by most of the children in both groups; just over 90% of them used this tool both at Time 1 and Time 2. On average, children worked with the magnifier for about two and a half minutes. At both times, children in both groups held the magnifying glass and looked through it. They looked at the objects in front of them and around the room and at the things they drew. As was suggested in Science Camp manual, all of the children were naturally interested in noticing how things looked; about half of the children in both groups commented about the fact that the magnifier made things bigger. At Time 2, children in the two groups used different terminology to describe more specifically what they saw through the magnifiers. A majority (87%) of children in the Treatment group referred to objects they worked with during the week, and talked about “tiny things” that “looked bigger.” One child, using words from one of the songs in the video clips said he used the magnifier “to see with your eyes” and he explained, “I used it before with all my friends and teachers.” When asked what they were doing and what they saw with the magnifier, a common answer among the Treatment group was, “Seeing something little to big. It looks big” In contrast, nearly half of the children in the Control group used terms such as “closer” and described looking for things, rather than looking at things. Several Control group children mentioned spying and looking for clues. Combined across the three key terms (Magnify, Tiny, Look Bigger), children in the Treatment group showed a statistically signficant increase in spontaneous use of terms at Time 2, compared to no change for the Control group. Within the range of 0 to 3 of the words, the average number of words used by the Treatment group, shown in Figure 2, reflects an increase of more than 100% from Time 1 to Time 2.

At Time 2, the average number of terms related to magnification more than doubled among the Treatment group. At Time 1, 21% of children used at least one term related to magnification; at Time 2, 40% did.


14

Figure 2 Children’s Spontaneous Use of Magnifying Terms: Magnify/Tiny/Look Bigger

N=210

Examples of children’s actions and comments, provided in Table 6, demonstrate the increased sophistication in use and understanding of the magnifier as a science tool among the Treatment group from Time 1 to Time 2. Descriptions in each row are from the same child, at Time 1 and Time 2.

Table 6 Treatment Group: Actions with the Magnifying Glass at Time 1 and Time 2

Time 1 Time 2 Looked through magnifying glass at pictures. C: "I can see a dinosaur and a Grandma." I: "What does it help with?" C: "It helps to see with the glass. It makes our eyes bigger."

Looks through magnifying glass at notebook and pictures. I: "What does it do?" C: "It lets you see bigger, my teacher told me that. I see it bigger and bigger! I: "What kind of things can you see?" C: "Can look at bugs and starfish. Big."

Used magnifying glass to look at popsicle sticks I: "How do you use it?" C: "At home I have a black one. You catch butterflies and let them go in the jungle."

Looks through magnifying glass. C: “Magnifier glass. I used it before at my school. We were science kids.”

He used the magnifying glass to look at the stickers.

He looked through the magnifying glass at various things, like his hand and the popsicle stick. I: "Can you tell me about what you're doing?" C: "I see bigger when I looked through it. I see my hand, can make it bigger. See little lines on my finger."

0.24

0.53

0.25 0.25

0

0.2

0.4

0.6

0.8

1

Time 1 time 2

Av

erg

e #

of

Wo

rd U

sed

Treatment Control


15

Charts

About three quarters of the children engaged in activity with the chart at Time 1, with a slight increase at Time 2 for both groups. On average, children worked with the charts for four to five minutes. At both times, children put stickers on and/or colored in the boxes of the chart in a methodical way; they lined the stickers up, and placed them in the middle of the cells. At Time 2, about one in five of the Treatment group children made a specific reference to the Science Camp activities, mostly by noting that their teacher told them the paper was called “a chart” and that it is used “to draw data.” When asked what they were doing, a handful of Treatment group children explained that they saw the same thing in the classroom and that their teacher told them “we see something and we draw it on the chart.” In contrast, children in the Control group said say they were “putting stickers” on it, “coloring,” and “drawing.” None of the Control group children used the word chart. Instead, they identified it as “paper” or “squares.” In reference to their own prior classroom work, a few Control group children referred to mixing colors, which was an activity one of the Control group teachers had described. Combined across the three key terms (Information, Chart, Data), children in the Treatment group showed a statistically signficant increase in spontaneous use of terms at Time 2, compared to their not using any such terms at Time 1 and compared no use at all among the Control group. Figure 3 shows the increase in use of chart-related terms among the children in the Treatment group only. On average, a small number of children used these terms; yet the increase was statistically significant, due to a complete absence of the terms at Time 1. Several children in the Treatment group used the words “chart” and “data” at Time 2. Figure 3 Number of Children in the Treatment Group Who Used Chart-Related Terms at Time 1 and Time 2

Time 1 Time 2

At Time 2, children in the Treatment group used the words “chart” and “data.” This reflected a significant change from Time 1, when no children in either group used these terms.

Chart

Chart

Chart Chart

Chart

Chart

Chart Chart

Data


16

Examples of children’s actions and comments, provided in Table 7, demonstrate the increased sophistication in use and understanding of the chart as a science tool among the Treatment group from Time 1 to Time 2. Table 7 Treatment Group: Actions with the Chart at Time 1 and Time 2

Time 1 Time 2 Drew dinosaurs. Made V and L with crayons. Traced hand on back of chart. C: "I can draw a pterodactyl. He has flat wings, spiky head. I can draw a brachiosaurus. He has a short head."

Drew pictures of dinosaurs on chart using stickers as a reference. I: "Do you know what this is called?" C: "This is called a chart, my teachers told me."

He went back and forth between coloring on the chart and doing a different activity. He said he was drawing a robot. He talked about "painting" with crayons in class during art. I: "What happens when you color on the chart with the crayons?" C: “I don’t know.”

He said he was drawing toys. He drew one item per box. C: “At my school. We see something and we draw it on the chart. All the shapes.”

Child put 1 sticker on the chart, somewhat hesitant. I: "What is it?" C: "It's a dinosaur. They're the same." [pointing to two of the same dinosaurs]

Drawing shapes and letters in boxes. Put stickers on chart. I: "Have you seen this before?" C: "I saw on the board with the shapes. I think it's called a chart. We talked about why Sid shrinks his shoes. It's in the story."

Measurement

About three quarters of the children engaged in measurement-related activity both at Time 1 and Time 2. On average, children worked with measurement for approximately two minutes. Children in both groups used the rulers to measure objects around them and referred to the numbers they saw on the rulers. They also used the rulers and the Popsicle sticks to align objects, to draw straight lines, and to count. At Time 2, children in the Treatment group focused more on measuring. More than half of them used the ruler to measure various things around them, including the charts, the boxes in the charts, and their own arms and legs. Children explained they could measure themselves and see “how big” or “how tall” different objects were in relation to other objects. One child identified the ruler as “a tool.” In contrast, while nearly half of the Control group children also referred to measuring and used words like “bigger” and “taller,” several of them found other uses for the materials such as tracing the ruler on a piece of paper, lining things up, drawing straight lines, and using the Popsicle stick as a nail file. In addition, while using the rulers and Popsicle sticks, several children in the Treatment group referred to “8,” which may reflect specific reference to the Science Camp non-standard measurement activity. The activity involved children


17

measuring different sides of a table or bookshelf with their hands. Suggested discussion questions included: “If the short side is 8 hands, how many hands will the long side be? Will it be more than 8? Fewer than 8?” Examples of comments among children in the Treatment group, both spontaneous and in response to interviewer questions included:

� "Seeing what it is. It's 8!"

� "To see how long, it's 8!"

� "I can measure myself with this! This crayon is 7. This book measures 8. My finger measures 5.”

Combined across the three key terms (Ruler, Non-Standard, Measure), children in the Treatment group showed a statistically signficant increase in spontaneous use of terms at Time 2, compared to no change for the Control group. Within the range of 0 to 3 of the words, the average number of words used by the Treatment group, shown in Figure 3, reflects an increase of more than 100% from Time 1 to Time 2. Figure 3 Children’s Spontaneous Use of Measurement-Related Terms: Ruler/Non Standard/Measure

N=210 Examples of children’s actions and comments, provided in Table 8, demonstrate the increased sophistication in use and understanding of the ruler as a science tool among the Treatment group from Time 1 to Time 2.

0.13

0.29

0.2

0.130

0.2

0.4

0.6

0.8

1

Time 1 Time 2

Av

era

ge

# o

f W

ord

s U

sed

Treatment Control

At Time 2, the average number of terms related to measurement more than doubled among the Treatment group. At Time 1, 13% of children used at least one term related to measurement; at Time 2, 24% did.


18

Table 8 Treatment Group: Actions with the Measurement at Time 1 and Time 2

Time 1 Time 2 Picked up ruler and measured his finger. C: “Oh my gosh. It's five feet tall!"

He measured his arm, using multiple ruler lengths. He counted 4½ ruler lengths. I: "Can you tell me about what you're doing?" C: "Measure."

Child places the ruler next to her finger, measuring it, and then 'high-fives' me with the illustrated hand that's on the ruler "Look my finger is five because I'm five years old."

Child uses ruler to measure various things. C: "I can measure myself with this!" [Holds ruler up to her arm. Holds up to magnifying glass] "It's smaller." [Holds ruler up to paper] "It's one!" [Holds ruler up to crayon] "This crayon is 7. This book measures 8. My finger measures 5."

Journal

Just over half of all children engaged in activity with the science journal, with a slight decrease at Time 2 for children in both groups. On average, children worked with the journals for just over three minutes. At both times, children in both groups spent most of their time drawing pictures of the stickers, and of themselves, their families, and the interviewer. Most were able to describe what they had drawn, when the interviewer asked. At Time 2, nearly one in five children in the Treatment group referred to the journal as a “scientist tool” or “my scientist journal” and noted that they, and scientists, use it “to observe” and to “write down” what they have done. None of the children in the Control group used the journal in any way other than to draw pictures. While a few children used terms specific to science journals at Time 2 (Observe, Journal, Draw), the change was an 18% increase from Time 1 and was not statistically significant or different from the children in the Control group. This corresponds with teachers’ reports about the success of their use of the journals during the Science Camp activities. Recall, more than half of the Treatment group teachers explained that children had trouble understanding the concept of recording their work in the journal, and that the journals were too small for easy writing. This difficulty in the classroom likely explains why fewer children replicated use of the journal during the one-on-one play activities. Examples of children’s actions and comments, provided in Table 9, demonstrate increased sophistication in use and understanding of the science journal as a science tool among the Treatment group from Time 1 to Time 2.


19

Table 9 Treatment Group: Actions with the Science Journals at Time 1 and Time 2

Time 1 Time 2 He drew a picture of himself and wrote his name on top

Drew me in the journal. C: “This is my scientist journal.”

Measured lines in the notebook using the ruler. Said she was writing her numbers.

Each time she did something in a box on the chart, she wrote random numbers down on the notebook page to represent what she had done on the chart. C: Pointed to chart and said, "I'm writing down what I did here."

Colored picture of her Mom. C: "I can't make it because the sun no make me see." [sun was blocking view of notebook]

Said she'd seen one before in class. She drew an “octopus with a lot of legs" and counted them. C: "You can write names and what you do in class. You write something and do stuff in your journal."

Children’s Actions with the Objects Provided While the number of distinct activities that children engaged in did not change over time, the number of different objects children used increased significantly at Time 2. At both times, children engaged in an average of three or four distinct activities (Mean = 3.48 activities; range = 1 to 6 different activities). Shown in Figure 4, at Time 1, children in both groups used three or four out of the seven different objects provided. At Time 2, children in the Treatment group used just over five objects, reflecting a 36% increase, compared to children in the Control group who used fewer than five of the objects (reflecting a 29% increase). The overall increase for all children, across both experimental groups, may reflect increased comfort with the research situation and/or with the interviewer. Figure 4 Number of Objects Used During Play Activities by Group: Time 1 and Time 2

N=220

3.79

5.17

3.79

4.90

0

1

2

3

4

5

6

7

Time 1 Time 2

Av

era

ge

# o

f O

bje

cts

Use

d

Treatment Control

Children in both groups used more of the objects provided at Time 2 than they did at Time 1.


20

Time Spent with Each Activity Overall, at both time points, children spent the longest time engaged in activity with the charts. On average, they spent between four and five minutes with the charts compared to an average of two to four minutes for the other activities. Among children in the Control group, average time spent on each of the four key activities decreased significantly at Time 2. This resulted in significant differences between the two groups at Time 2, such that children in the Treatment group spent more time, on average, engaged in activities with the science journals, measurement tools, and magnifiers than did their counterparts (See Figure 5). Figure 5 Time in Minutes Children Spent Engaged in Activities at Time 2

N=130 N=134 N=187

Awareness of Science in their Lives At Time 2, most children in both groups reported that they knew a scientist; about half in each group reported they themselves were scientists. In the Treatment group, more of the children referred to “Sid the Science Kid,” Sid’s friends, or someone they knew in real life, whereas the majority of children in the Control group said “yes” they knew a scientist, but did not refer to anyone specifically. A few of the Control group children in Los Angeles referred to having seen Sid “yesterday, in my classroom,” because the final Science Camp celebration took place at their school before their final data collection sessions. They knew that Sid visited the school and they “high fived” him. However, they did not refer to any of the science tools or activities related to the curriculum.

3.62

2.98

2.522.55

1.611.78

0

1

2

3

4

Journal Measurement Magnification

Treatment Control

At Time 2, children in the Treatment group engaged with each activity longer than did children in the Control group. Children in the Control group appeared less interested in exploring the science tools for a second time.

Av

era

ge

# o

f m

inu

tes


21

Examples of Treatment group responses included:

� “Yes, Sid wanted to measure a big blue whale. Sid the Science Kid visited our class yesterday.”

� “Sid the scientist kid when we were watching yesterday.”

� “Sid is a scientist, and we watched the show.” In explanation of what scientists do, more than half of the children in the Treatment group referred to Science Camp activities they had conducted and one third referred to scientists’ use of “scientific tools.”

“Like use this paper so we can collect data.” “They just use these tools, like a magnifier and a measurement. They use a stick to measure this and they can measure with a crayon. “

One third of children in the Control group said they did not know what scientists do and 25% of them gave an answer that was not related to science including, “ they play,” “ they color,” and “they eat fruit.” Generally, after one week of Sid Science Camp activities and video, children in the Treatment group were able to make real-world connections with the activities they conducted in their classes. They showed interest in the activities they had done, and excitement about continuing to try out those activities on their own time. Treatment group teachers believed their students showed more interest in science learning as a result of the Science Camp activities. In support of what we learned from the students’ behavior as well as what we observed in Treatment group classrooms, after using the Science Camp curriculum, seven of the eight Treatment group teachers perceived that their students showed a lot to a great deal more interest in science learning. One teacher explained, “The children repeated the activities during free choice time, weeks later - measuring objects, making their own charts, etc.” During on-site visits during the week of the intervention, GRG researchers’ observations confirmed this, as students were actively engaged with the magnifiers and rulers during free-play time. In both locations, children spoke about Sid and his friends as they used their own magnifiers to look at small objects in and outside the classroom. In New York, children offered to share their “science” magnifying glass with the interviewer, with the caution to treat it gently because it was so special. In Los Angeles, during outside play, a child wanted to examine through a magnifying glass a snail the teacher had just found. This type of behavior supports the finding that the Science Camp activities effectively engaged and interested the kids, as they

After completing the Science Camp week, Treatment group children seemed more attuned to the science around them, and were able to verbalize their understanding of science tools.


22

transferred their knowledge of science tools and used them accurately, outside of the curriculum activities. Seven of the eight Treatment group teachers indicated that students learned about the purpose and use of scientific tools as well as new science-related vocabulary. Examples of teachers’ descriptions of student learning included:

� “That a magnifying glass makes things look bigger. We use charts to record and share information. We use rulers to measure.”

� “The names and what you can use them for and when you can use them.”

� “They learned the meaning of the word. Now they know that magnify means make things look bigger, that chart is the same as graph and that [it is used] to collect data.”

After the Science Camp activities, teachers believed that their students learned about the purpose and use of the scientific tools and science-related vocabulary.


23

TEACHERS’ KNOWLEDGE AND COMFORT: CHANGES SEEN OVER TIME Teachers themselves learned new science content and expanded their views about science in general and in the classroom. Nearly half of the Treatment group teachers described that they themselves learned most about non-standard measurement. They learned, “that you can use hands, feet, and other objects for non-standard measurement and only 1 item can be used at a time.” After completing the Science Camp, all eight Treatment group teachers reported they considered more activities to be “science activities” than they did before the week. The majority also expressed more confidence in using science vocabulary with their students, encouraging students to seek out their own answers to questions, and using TV as an educational tool. See Table 10. Table 10 Treatment Group Teachers’ Changes in Science Attitudes after the Science Camp More

than before

About the same

Less than

before The range of activities I would consider “a science activity.”

8 – –

I’m likely to use science vocabulary in conversations with kids in my classroom.

7 1 –

I’m confident in my ability to use television as an educational tool.

5 3 –

I’m likely to encourage kids to seek answers to their own questions

5 1 –

I’m confident in my ability to conduct science activities with kids in my classroom.

4 3 1

I’m interested in doing science activities with kids in my classroom.

4 4 –

I’m likely to incorporate science into our regular classroom routine.

4 4 –

I’m likely to encourage kids to explore the world around them.

4 3 1

I’m comfortable doing science activities with kids in my classroom.

3 4 1

N=8

Half of the Treatment group teachers increased their interest, likelihood, and confidence to incorporate more science into their classroom routines. Nearly all of them increased their likelihood to use science terms in their conversations with children and broadened their perspective of what they consider to be “science.”


24

Teachers in the Treatment group perceived knowledge gains and increased confidence with science content after conducting the Sid Science Camp with their students. Compared to those in the Control group, Treatment group teachers’ perceptions of their own knowledge of science content for pre-school aged children increased after they conducted the Science Camp activities for one week. At Time 1:

� 2 teachers (both in the Treatment group) knew not much at all about pre-school science content

� 14 teachers (6 in the Treatment group; 8 in the Control group) knew a fair amount

At Time 2:

� 12 teachers (6 in each group) knew a fair amount � 4 teachers (2 in each group) knew a lot

On average, teachers in the Treatment group rated themselves lower at Time 1 than did teachers in the Control group. Subsequently, they showed a larger increase in their perceived knowledge at Time 2, as shown in Figure 6. Figure 6 Teachers’ Perceived Knowledge of Preschool Science Content: Time 1 and Time 2

N=16 Scale: 1=I don’t know very much at all; 2=I know a little bit; 3=I know a fair amount; 4=I know a lot; 5=I am an expert in this area

2.43

3.293.00 3.22

1.00

2.00

3.00

4.00

5.00

Time 1 Time 2

Pe

rce

ive

d K

no

wle

dg

e

Treatment Control

After one week with the Science Camp activities, teachers showed increased perceived knowledge of science content, a broadened view about what it means to do science in preschool classrooms, and increased comfort with a variety of science teaching strategies.


25

Teachers also reported slight changes in how they responded to students’ science-related questions after the intervention week. Shown in Table 11, at Time 1 there were few differences between the groups. At Time 2, more of the Treatment group teachers reported looking for answers in books or on websites (by themselves or with the child), and suggesting that the child asks someone else (e.g., parents, older siblings, an expert). The shift toward looking for answers along with the child, seen in the Treatment group more than in the Control group, suggests an increased confidence about not having to provide the child with an immediate answer. This approach parallels the goals of the STSK program, to encourage a climate of curiosity among children and adults. Table 11 How Teachers Respond to Students’ Science-Related Questions

Treatment Control Time 1 Time 2 Time 1 Time 2

Try to answer as best I can 7 8 6 8 Look it up in a book, myself 6 8 6 6 Look it up on a website, myself 6 8 4 5 Look it up in a book together with the child 6 8 5 6 Look it up on a website together with the child 2 5 2 1 Suggest he/she asks someone else. 2 4 1 0

N=16 Teachers in the Treatment group showed increased comfort with science teaching strategies after conducting the Sid Science Camp with students. Before the Science Camp activities, teachers’ ratings of comfort with various science activities in their classroom revealed no differences between groups. After the intervention week, Treatment group teachers rated their comfort higher than did teachers in the Control group on several items:

� Incorporating science into circle time activities � Incorporating science into small group activities � Incorporating science into the free play options in the

classroom � Incorporating science into the free play options outside � Reading books about science topics � Teaching language and literacy during science activities � Teaching math during science activities � Asking children open-ended questions during hands-on

science activities � Encouraging children to reflect on their hands-on science

experiences

Even with a similar approach to their regular science teaching, Treatment group teachers reported higher comfort with several science-teaching strategies after completing one week of the Sid Science Camp.

At Time 2, Treatment group teachers were more comfortable to look together with a child for answers to science-related questions.


26

Additionally, for the Treatment group only, comfort with two items increased significantly from Time 1 to Time 2:

� Incorporating science into the free play options outside � Encouraging children to reflect on their hands-on science experiences

Table 12 shows average ratings on a scale from 1 (Not at all comfortable) to 5 (Extremely comfortable). Recall that the educational philosophy and teaching approach for the Science Camp did not differ from teachers’ typical teaching; further evidence for the value added by using video, music, and lively characters as models of science exploration and learning. Table 12 Teachers’ Mean Comfort at Time 1 and Time 2 Time 1 Time 2

Treatment Control Treatment Control Trying new materials or activities yourself before using them with children

3.86 3.88 4.71 4.57

Incorporating science into circle time activities a 4.13 4.38 4.75 4.00 Incorporating science into small group activities a 4.50 4.50 4.75 4.14 Incorporating science into the free play options in the classroom ab

4.13 4.25 4.63 3.88

Incorporating science into the free play options outside a

3.57 4.14 4.63 3.50

Reading books about science topics a 5.00 4.43 5.00 4.00 Teaching language and literacy during science activities a

4.63 4.25 4.75 3.75

Teaching math during science activities a 4.38 4.38 4.71 3.88 Using related video when teaching a science topic

3.00 4.00 3.88 3.80

Getting children excited about science 3.88 4.25 4.50 4.00 Guiding children in hands-on science activities 4.13 4.38 4.63 4.13 Asking children open-ended questions during hands-on science activities a

3.75 4.00 4.75 3.75

Encouraging children to reflect on their hands-on science experiences a

4.14 4.25 4.88 4.00

Asking children to share their discoveries with each other during science activities

4.43 4.38 4.75 4.25

Responding when a child asks a science-related question and you don’t know the answer

3.29 3.57 3.63 3.38

N=16 a Treatment group rating was higher than Control group rating at Time 2. ab Treatment Group Time 2 rating is higher than Time 1. At both Time 1 and Time 2, teachers in both groups reported experiencing relatively few challenges when leading hands-on science activities with their students. At Time 2, one item, classroom time constraints, was considered more


27

of a challenge among Control group teachers than Treatment group teachers. (See Appendix A for teachers’ ratings of each challenge). TREATMENT GROUP TEACHERS’ SATISFACTION Overall, Treatment group teachers were very satisfied with the Sid Science Camp curriculum including the one-day training session, the ease of implementation, and the ease with which the activities fit into their regular daily schedules. After the intervention week, Treatment group teachers reported they were very satisfied with the training received before they conducted the Science Camp. They found it extremely useful for implementing the curriculum. Teachers explained: “I think the training is an excellent resource that helped me use the material successfully.” Another teacher suggested expanding the training period beyond one day. “I think the training was very useful. It would have been different if we weren't trained. I would have felt ill-prepared and unfamiliar with the materials and concepts. Maybe 1 more day of training would be helpful.” During the onsite visit in Los Angeles, one teacher expressed a preference for training more in advance of the start date. She believed the trainers were impressive and she felt prepared to lead the activities with her students. However, she would have preferred more time to review and try out all the activities in the manual. Given more time, she would have tried to collect more materials that matched the activities conducted in the video clips. Treatment group teachers were able to integrate the Science Camp activities easily in their classroom over the week. As shown in Table 13, all Treatment teachers reported that most or all of the time:

� materials needed for activities were already available in their classrooms, � they were able to find space to set up the activities, and � the activity directions were clear.

Table 13 Teachers’ Experiences Preparing and Implementing the Curriculum None of

the time Some of the time

Most of the time

All of the time

Were the materials needed for the activities easy to find and/or already in your classroom?

– – 2 6

Were you able to find space in your class to set up the activities?

– – 1 7

Were the directions for the activities clear and easy to understand?

– – 1 7

Did you try the activities on your own before having your children try them?

1 4 – 3

Did you make changes to the activities?

4 3 1 –

N=8


28

The Science Camp activities were easy for teachers to incorporate into their curriculum because the directions were clear and the scheduled time for each day was easy to follow (See Table 14). The majority of teachers reported that the number and variety of activities included in the Science Camp curriculum were just right. Table 14 Teachers’ Ratings of Ease of Use Mean

(1-5) How easy was it to incorporate the Sid the Science Kid activities into your curriculum?

4.50

How easy was it to follow the activity directions? 4.50 How easy was it to follow the scheduled time each day? 4.25 N=8 Scale: 1 (Not at all) to 5 (Extremely) Table 15 shows that teachers were quite satisfied with all elements of the curriculum. Average ratings on a scale from 1 (Not at all satisfied) to 5 (Extremely satisfied) were all higher than 4 out of 5. In particular, all eight teachers were extremely satisfied with the tip-sheets for parents, and the take home science tools Table 15 Teachers’ Satisfaction with the Science Camp Curriculum Mean

(1-5) The Tip Sheets for Parents 5.00 The take-home science tools (magnifying glasses, journals, stickers) 5.00 The videos 4.88 The ease of using the materials in the classroom 4.75 The format of the curriculum 4.63 The teacher preparation information 4.63 The activities 4.25 The week-long schedule 4.13 N=8 All eight teachers would be extremely likely to recommend both the Science Camp curriculum and training to friends and colleagues. All were either extremely (n=7) or very (n=1) likely to recommend the STSK television series program to colleagues.


29

Table 15 Teachers’ Likelihood to Recommend the Program to Others Mean

(1-5) Recommend using the Sid the Science Kid curriculum to colleagues for use in their classrooms.

5.00

Recommend attending a Sid the Science Kid workshop to friends or colleagues.

5.00

Recommend viewing Sid the Science Kid to friends or colleagues.

4.88

N=8


30

CONCLUSIONS Based on these findings, GRG concludes that the Sid the Science Kid Science Camp curriculum effectively enhanced both students’ and teachers’ understanding of the content. After one week of Science Camp activities, children showed increased understanding of science tools and their functions, and demonstrated more sophisticated interactions with and descriptions of the tools. Even with a similar teaching approach already in use, after one week with the Science Camp activities, teachers showed increased perceived knowledge of science content, a broadened view about what it means to do science in preschool classrooms, and increased comfort with a variety of science teaching strategies. The evaluation of this one-week pilot curriculum supports there was a clear added value of the use of educational video in the classroom. Using characters (to which children positively related) to teach, convey, and demonstrate science learning was an effective strategy. Beyond learning about science tools, both the teachers and students had enjoyed the experience. Findings that address the main research questions follow: The Science Camp curriculum was quite effective in teaching preschool students the specific science content included in the week-long unit. After one week with the Sid Science Camp activities, watching the video clips and conducting and recording their own science investigations, children acquired an understanding of science, scientists, and the function of various science tools. During the play activities with interviewers, children in the Treatment group were able to make connections to the activities that were conducted in their classroom during the week of the intervention, in a way that Control group children were not. Moreover, they were better able to express their understanding verbally to the interviewers than were their counterparts. Short term effects of the Science Camp on students were seen in increased content knowledge, new science terminology integrated into their vocabulary, and more sophisticated understanding of what science is and how science tools are used. Students’ spontaneous and accurate use of science-tool related terminology more than doubled after one week of exposure to the new vocabulary and accompanying activities. Compared to their actions before the week and compared to the Control group, after the Science Camp children interacted with the objects in a way that showed understanding and exploration of the new terms and concepts they had learned, particularly for measurement and magnification.


31

Teachers found the curriculum and its activities to be easy to use. They felt prepared to conduct the activities after their one-day in-person training, and after a week of use, teachers saw changes in their own and their students’ science content knowledge. Teachers were, overall, quite satisfied with the Science Camp training session and found it useful preparation to implement the curriculum in their classrooms. During the week, teachers had no difficulty acquiring the necessary materials, scheduling the appropriate time for activities, and conducting the activities with their students. Aside from some difficulty with recording their own observations in the science journals, teachers found that the tools and activities effectively engaged their students. The inclusion of STSK video clips was a successful way to introduce the activities. GRG researchers observed a high level of energy and enthusiasm among teachers and students while watching the videos and conducting the classroom activities. Students saw the characters having fun with science, and they, in turn had fun conducting the activities themselves.

RECOMMENDATIONS GRG recommends providing teachers with more options and suggestions for ways to extend the curriculum in classrooms. A few teachers mentioned that a week was too short for the implementation of the curriculum. Teachers would like to dedicate two or more days to each activity to ensure that students grasp the concept of using different scientific tools. Highlight in the teachers’ manual the ideas for extending each lesson with additional activities. For example, add a chart for each theme that shows teachers how to conduct the lessons if they have two days, five days, or up to 10 days in which to implement the activities. Given flexibility to expand the activities over time, teachers can tailor the curriculum to fit their students’ needs. Extended use of the curriculum may increase the likelihood of longer-term positive effects on the students. GRG recommends spending more time on -- and including activities specifically dedicated to -- the use of science journals to record science investigations and observations. Several teachers indicated that the journal activity was the least successful as many students did not understand the idea of recording their own observations on paper. GRG researchers noted that students struggled with recording their own observations, and in their one-on-one play activities, children did not use the science journals as much as they did other science tools provided. Specific activities focused solely on this science tool may help children to understand the concept. Children are likely to benefit from more specific practice.


32

GRG recommends more repeated use of and emphasis on general science terms, as well as those terms and phrases specific to each lesson. The words that children heard many times and that had specific activities associated with them were the words that they incorporated into their vocabularies (e.g., magnify, measure). In order for students to learn science words such as “compare,” “explore,” and “observe,” those words should be more prevalent throughout the lessons. GRG makes the following recommendations for future research and evaluation of the Sid Science Camp program:

� Seek out and recruit for participation teachers who do not have science teaching experience and background that matches the Science Camp curriculum as closely. It will be useful to learn to what extent teachers can implement these activities and incorporate them in their classroom if they are not already accustomed to the approach.

� Include teachers from multiple schools in one district, if possible, and assign teachers in different schools to the different experimental conditions. Despite teachers’ understanding of the importance of not discussing the curriculum with others (i.e., teachers in the Control group), in one school building there will likely be some discussion and sharing of information about the curriculum. Moreover, teachers whose own children are students in one or another classroom may be inadvertently exposed to the intervention.

Overall, given the short duration of the intervention (one week), and the short period of time between the first and second data collection, the current evaluation revealed the Science Camp curriculum was quite effective. As it is further developed, with additional lessons added, there is potential for a very successful early childhood science program.

Goodman Research Group, Inc. Main Office 929 Massachusetts Avenue, Suite 2A Cambridge, Massachusetts 02139 Tel: (617) 491-7033 Fax: (617) 864-2399 [email protected] www.grginc.com © 2012 Goodman Research Group, Inc.

Appendix A: Copies of All Survey Instruments

2

Teacher Pre-Survey with Data Summer Science:

Pre-survey FAMILIARITY WITH SID THE SCIENCE KID

1. Are you familiar with Sid the Science Kid? (Check only one.)

Respondents Yes 9 No 6

N= 15 2. I f yes: With which aspects of Sid the Science Kid are you familiar? (Check all that apply.)

Respondents I’ve watched it on TV 6 I’ve watched it on the web (www.pbskids.org/sid) 2 Other 1 I’ve watched in on YouTube.com – I’ve played games on the SID website – I’ve done activities suggested on the SID website – I’ve printed out activity or coloring pages from the SID website – N= 9

� I watched the videos given by the program

3. Have you ever attended a training about how to conduct science activities in the classroom? (Check all that apply.)

Respondents No 8 Yes, a training at my center 6 Yes, a training at a conference 1 Yes, via webinar –

N= 15 For the purposes of this annotated survey, questions 4 through 15 include data from only the head teacher in each classroom. Information from teaching assistants was excluded. YOUR CLASSROOM

4. How many children are in your classroom?

Average Range Girls 16 7-25 Boys 17 6-25

N= 10

5. What race(s)/ethnicities are the children in your classroom? (Write the approximate number of children in each blank)

Average Range Hispanic or Latino 27 0-43 Asian 2 0-5 Black or African American 1 0-3 White 1 0-5 American Indian or Alaska Native 0 - Native Hawaiian or Other Pacific Islander 0 - Other; please specify 0 -

N= 6

6. How many children in your classroom are English language learners?

� Average: 16 � Range: 1-32

7. How many children in your classroom speak a language other than English at home?

� Average: 26 � Range: 14-39

8. What languages do these include?

Total Spanish 6 Tagalo 2 Other 3

N=10 Note: Total exceeds 10 as respondents were able to select multiple options.

9. How many classroom assistants do you have, if any? (Check only one.)

Number of respondents

None 1 One 3 Two 1 Three or More 1

N= 6

10. How old are the children in your classroom?

Average Range 3 years old or younger <1 0-2 4 years old 18 1-47 5 years old 1.5 0-10

N= 5

4

11. Which of the following do you have access to for use in your classroom? (Check all that apply.)

Number of Respondents

DVD Player 6 Personal Computer 3 Television 4 Internet connection 4 Printer 4 SMART Board/Interactive White Board 2 VCR –

N= 10 Note: Total exceeds 10 as respondents were able to select multiple options.

YOUR CLASSROOM PRACTICES

12. In your classroom do you use a particular core cur r iculum?


Yes 9 No 1

N=10

13. I f yes, what is the name of the Curr iculum?

� Creative Curriculum N=9

14. Does the core cur r iculum include science? (Check only one.)


Yes 10 N=10

15. Do you use any additional science cur r iculum? (Check all that apply.)


Other, please list 5 No additional science curriculum 3 Science Literacy Centers 2 AIMS – FOSS Kits – Galileo Resources – GEMS – Mudpies to Magnets – Young Scientist Series –

N=10

� Creative curriculum � Zula International

YOUR EXPERIENCE WITH TEACHING SCIENCE

16. How do you, or would you, respond when a student asks you a question about science? (Check all that apply.)


Try to answer as best I can 13 Look it up in a book, myself 12 Look it up in a book together with the child 11 Look it up on a website, myself 10 Look it up on a website together with the child 4 Suggest he/she asks someone else. 3 Other 3

N=16 Note: Total exceeds 10 as respondents were able to select multiple options

17. I f suggest he/she asks someone else who?

� Depends on the question. An expert � Parents of other teachers around � Parents or older siblings

18. I f other , please descr ibe:

� Any family member � Explain that we will find the answer

19 . How would you rate yourself in terms of your knowledge of science content for preschool-aged children? (Check only one.)

Mean (1-5)

I don’ t know very much at all (1)

I know a little bit (2)

I know a fair amount

(3)

I know a lot (4)

I am an exper t in this area (5)

2.75 2 – 14 – – N=16 20. Which of the following descr ibe the ways you approach teaching science in your classroom? (Check all that apply.)


Small group activities 14 Kids do almost all hands- on activities 11 Large group activities 10 Kids do some hands-on exploration 6 I lead science activities; kids watch as I handle the materials 3 Kids do very little hands-on exploration 1


6

21. Where do kids in your classroom go to do science activities? (Check all that apply.)


Designated science area in the classroom (for example, a science table) 14 Designated science area outside of the classroom 6 No designated science area; science activities can be done anywhere in the classroom 4 Several different designated science areas throughout the classroom 3 Other; Please describe 1

N=16 Note: Total exceeds 16 as respondents were able to select multiple options. 22. I f other , please descr ibe:

� The park

23. Which of the following descr ibe ways that science is conducted in your classroom? (Check all that apply.)


Kids work in small groups, some doing science activities and some doing a different activity 15 All kids have the choice to do a science activity; some choose to participate and some do not 10 All of the kids do a science activity at the same time, in small groups 4 All of the kids do a science activity at the same time, in a large group 2


24. On average, how often is science available in your classroom throughout the school year? (Check only one.)


Every few months – Every other month – Once a month, every month – A few times a month 3 Every week 2 Every day 11

N=16

25. Dur ing a typical week when you are doing science with your kids, how many days in that week is science offered? (Check only one.)


One day 2 Two days 1 Three days 1 Four days 1 Five days (everyday) 9

N=14

26. Have you done any of the following activities in your classroom? � For each, indicate whether you have done this or not (Yes/No).

Yes No Trying new materials or activities yourself before using them with children

15 –

Incorporating science into circle time activities 16 – Incorporating science into small group activities

16 –

Incorporating science into the free play options in the classroom

16 –

Incorporating science into the free play options outside

14 1

Reading books about science topics 15 – Teaching language and literacy during science activities

16 –

Teaching math during science activities 16 Using related video when teaching a science topic

5 10

Getting children excited about science 16 – Guiding children in hands-on science activities 16 – Asking children open-ended questions during hands-on science activities

16 –

Encouraging children to reflect on their hands-on science experiences

16 –


15 1

Responding when a child asks a science-related question and you don’ t know the answer

14 1

N=15-16 27. How comfortable are you doing the following in your classroom? � I f you have, then rate your comfort:

Mean (1-5)

Not at All (1)

A L ittle

(2)

Somewhat

(3)

Very (4)

Extremely

(5) Trying new materials or activities yourself before using them with children

3.87 – 1 5 4 5

Incorporating science into circle time activities 4.25 – – 3 6 7 Incorporating science into small group activities 4.50 – – – 8 8 Incorporating science into the free play options in the classroom

4.19 – 1 4 2 9

Incorporating science into the free play options outside 3.86 – 1 5 3 5 Reading books about science topics 4.73 – – 1 2 12 Teaching language and literacy during science activities 4.44 – – 2 5 9 Teaching math during science activities 4.38 – – 2 6 8 Using related video when teaching a science topic 3.56 1 1 1 4 2 Getting children excited about science 4.06 – – 4 7 5 Guiding children in hands-on science activities 4.25 – – 3 6 7 Asking children open-ended questions during hands-on science activities

3.88 – – 4 10 2


4.20 – – 3 6 6


4.40 – – 2 5 8


3.43 – 1 8 3 2

N= 8-16

8

28. How much of a challenge are the following when leading hands-on science activities with children in your classroom?

Mean (1-5)

Not at All (1)

A L ittle

(2)

Somewhat (3)

Very (4)

Extremely (5)

Classroom time constraints 2.38 4 2 10 – – Classroom space constraints 2.00 5 4 5 – – Lack of necessary materials for activities 2.19 6 5 3 – 2 Lack of necessary equipment 2.00 7 6 1 – 2 Lack of necessary teacher resources 1.80 10 2 1 – 2 Needing to meet my center’s standards and requirements

1.63 9 4 3 – –

Difficulty integrating supplemental science into our core curriculum

1.53 8 6 1 – –

Difficulty getting children excited about science 1.44 11 3 2 – – My own comfort with science 2.19 6 3 5 2 – Students’ language barriers 2.06 5 5 6 – – Students’ lack of interest in particular topics 2.38 4 2 10 – – Students’ discipline issues 2.69 3 2 9 1 1 Parents’ wishes for student outcomes 1.56 12 1 1 2 – Other challenge 3.00 1 – – – 1

N= 14-16

29. FOR RESPONDENTS WHO SELECTED OTHER: Please descr ibe: � Motivating parents to work with their children at home

YOUR INTERACTION WITH PARENTS

28. In which of the following ways, if any, do you share information about your program with parents? (Check all that apply.)


On a bulletin board in your classroom 15 In person during drop off or pick-up 14 During scheduled one-on-one conferences with parents 14 Through a newsletter 13 During regular group meetings with parents 10 During parents’ nights/workshops 8 Use parent-leaders to share information with other parents 8 By phone 4 By email 3 Other 3

N=16 29. FOR RESPONDENTS WHO SELECTED OTHER: Please descr ibe:

� During monthly classroom meetings

30. Do you provide any of the above in your students’ home language (other than English)? (Check only one.)

Number of Respondents Yes 13 No, but I am interested in doing so 2 No, and I am not interested in doing so – Does not apply; all of my students speak English –

N=15

31. Which of the following do you cur rently do to encourage parents to suppor t their children’s science-related exper iences at home? (Check all that apply.)


Give parents ideas for science activities they can do with their children 10 Give parents take-homes to read (such as one-page tip sheets or fact sheets) on science activities 8 Hold parent workshops 4 Share science-related books 2 Share links to websites with science-related information and/or video 2 Share science-related educational videos 1 Other 1

N=16 Note: Total exceeds 16 as respondents were able to select multiple options. 32. FOR PARTICIPANTS WHO SELECTED OTHER: Please descr ibe:

� Giving parents suggestion to get their children involved in daily life activities like trips to the supermarket, cooking, talking about the weather etc.

The following background questions are for descr iptive purposes only.

33. Are you: Number of

Respondents Female 16 Male –

N=16 34. What is your race/ethnicity? (Check all that apply.)


Latino/a or Hispanic 11 White 2 Black or African-American 2 Prefer not to respond 1 American Indian or Alaskan Native – Asian – Native Hawaiian or Other Pacific Islander –

N=16

10

35. What is the last level of education you fully completed? (Check only one.) Number of

Respondents Some high school 1 High school diploma or GED - Some college/trade school - Associate’s (2 year) college degree 5 Bachelor’s (4 year) college degree 9 Master’s degree 1 Multiple master’s degrees - Doctorate/other post graduate -

N=16 36. How many years have you been an ear ly childhood teacher?


One to Five 2 Six to Ten 6 Eleven to Fifteen 6 Fifteen or More 1

N=15 Average: 10 years Range: 2-18 years

Teacher Post-Survey with Data

Summer Science:

Post-survey Please think about the week-long Sid the Science K id Science Camp activities you completed and tell us what you thought, and what you would suggest to improve the exper ience. 1. How satisfied were you with the following elements of the Sid the Science Kid science camp:

Mean (1-5)

Not at All (1)

A L ittle

(2)

Somewhat

(3)

Very (4)

Extremely

(5) The format of the curriculum 4.63 0 0 0 3 5 The week-long schedule 4.13 0 0 2 3 3 The ease of using the materials in the classroom 4.75 0 0 0 2 6 The teacher preparation information 4.63 0 0 1 1 6 The activities 4.25 0 0 1 4 3 The videos 4.88 0 0 0 1 7 The Tip Sheets for Parents 5.00 0 0 0 0 8 The take-home science tools (magnifying glasses, journals, stickers)

5.00 0 0 0 0 8

N=8 2.

Just Right (1)

Too much (2)

Not enough (3)

What did you think of the var iety of activities that were included? 5 0 2 What did you think of the number of activities that were included? 5 0 2

N=7

3. Rate the Sid the Science Kid cur r iculum mater ials on the following dimensions: (Circle one per row.)

How easy was it to: Mean (1-5)

Not at all

Easy (1)

(2)

(3)

(4)

Extremely Easy (5)

follow the activity directions? 4.50 0 0 0 4 4 follow the scheduled time each day? 4.25 0 0 1 4 3 incorporate the Sid the Science Kid activities into your curriculum?

4.50 0 0 1 2 5

N=8

12

4. For each question, mark the box that descr ibes how often you exper ienced it. (Circle one per row.)

How often… Mean (0-3)

None of the time

(0)

Some of the time

(1)

Most of the time

(2)

All of the time (3)

Were the materials needed for the activities easy to find and/or already in your classroom?

2.75 0 0 2 6

Were you able to find space in your class to set up the activities?

2.88 0 0 1 7

Were the directions for the activities clear and easy to understand?

2.88 0 0 1 7

Did you try the activities on your own before having your children try them?

1.63 1 4 0 3

Did you make changes to the activities? 0.63 4 3 1 0 N=8 SID THE SCIENCE KID ACTIVITIES 5. Which of the activities did you try with your students dur ing the week that you used the Sid the Science

K id cur r iculum? (Check one per row.)

Yes No Use magnifiers in science explorations. 15 1 Record observations in journals. 11 5 Collect data and record it on a chart. 12 4 Activities about linear measurement. 12 4 Discussions about linear measurement. 9 7 Activities about non-standard measurement. 12 4 Activities about standard-measurement. 12 3

N=15-16 6. Which activities were most successful? (Check all that apply.)


Use magnifiers in science explorations. 9 Activities about non-standard measurement. 9 Collect data and record it on a chart. 7 Activities about standard-measurement. 5 Record observations in journals. 3 Activities about linear measurement. 2 Discussions about linear measurement. 2 Growing seeds 2 Rock candy/growing crystals 2 Farms 1 Mixing colors 1

7. Please explain. Click here for a complete list of responses

8. Which activities were least successful? (Check all that apply.)


Record observations in journals. 6 Activities about linear measurement. 3 Fossils 3 Discussions about linear measurement. 2 Collect data and record it on a chart. 1 Activities about non-standard measurement. 1 Sea animals 1 Farms 1 Use magnifiers in science explorations. 0 Activities about standard-measurement. 0

9. Please explain: Click here for a complete list of responses

10. I f you didn’ t try some of the activities, please share your reason(s). � I didn't consider them age appropriate

11. How do you think your use of the Sid the Science Kid cur r iculum mater ials would have been different if you didn’ t attend the training?

Click here for a complete list of responses SID THE SCIENCE KID VIDEOS

12. Which video segments did you view with your students? (Check all that apply.) Number of Respondents SID program segments that were provided 8 Fab Lab segments that were provided 7 Additional segments that were not provided (for example, segments from the web) 1 I did not use video segments with my students 0

N=8

13. Descr ibe an example of how you used the video segments. Click for a complete list of responses 14. I f you showed video segments, which type did you prefer? (Check only one.)


I thought both were equally good. 8 I preferred to show the animated program segments 0 I preferred to show the Fab Lab live-action clips. 0

N=8

15. I f you did not use any of the video segments in class, please tell us why not.

� N/A

14

FAMILY TAKE-HOME MATERIALS

16. Which, if any, did you send home with the children? (Check one per row.)

Yes Sid the Science Kid videos 8 Tip Sheets for Parents 8 Tips Sheets in Spanish. 8 Magnifying glass 8 Science Journal 8 Sid the Science Kid stickers 7 Sid the Science Kid poster 7

N=8 17. What feedback, if any, did you get from parents regarding the family take-home mater ials?

Click here for a complete list of responses

YOUR SCIENCE TEACHING 18. How do you, or would you, respond when a student asks you a question about science? (Check all that apply.)


Try to answer as best I can 16 Look it up in a book, myself 14 Look it up on a website, myself 13 Look it up in a book together with the child 14 Look it up on a website together with the child 6 Suggest he/she asks someone else. 4 Other 1


19. FOR RESPONDENTS WHO SELECTED SUGGEST HE/SHE ASKS SOMEONE ELSE: Who? � an expert � another teacher � parent, older siblings

20. FOR RESPONDENTS WHO SELECTED OTHER: Please descr ibe

� Maybe as a group at circle time or rug time. Ask the question and see what other children thought and try to get an answer.

21. How would you rate yourself in terms of your knowledge of science content for preschool-aged children?

(Check only one.)

Mean (1-5)

I don’ t know very much

at all (1)

I know a little bit

(2)

I know a fair amount

(3)

I know a lot (4)

I am an exper t in this area

(5)

3.25 0 0 12 4 – N=16

22. How comfortable are you doing the following in your classroom? � For each, indicate whether you have done this or not (Yes/No).

Yes No Trying new materials or activities yourself before using them with children

14 2

Incorporating science into circle time activities 16 0 Incorporating science into small group activities

16 0

Incorporating science into the free play options in the classroom

16 0

Incorporating science into the free play options outside

14 2

Reading books about science topics 16 0 Teaching language and literacy during science activities

16 0

Teaching math during science activities 15 0 Using related video when teaching a science topic

8 7

Getting children excited about science 16 0 Guiding children in hands-on science activities 16 0 Asking children open-ended questions during hands-on science activities

16 0


16 0


16 0


13 2

N=15-16

23. How comfortable are you doing the following in your classroom? � If you have, then rate your comfort:

Mean (1-5)

Not at All (1)

A L ittle

(2)

Somewhat (3)

Very (4)

Extremely (5)

Trying new materials or activities yourself before using them with children

4.64 0 0 0 5 9

Incorporating science into circle time activities 4.38 0 0 1 8 7 Incorporating science into small group activities 4.47 0 0 0 8 7 Incorporating science into the free play options in the classroom

4.25 0 0 3 6 7

Incorporating science into the free play options outside 4.06 0 1 3 6 6 Reading books about science topics 4.50 0 0 0 8 8 Teaching language and literacy during science activities 4.25 0 0 3 6 7 Teaching math during science activities 4.27 0 1 0 8 6 Using related video when teaching a science topic 3.85 1 1 1 6 4 Getting children excited about science 4.27 0 1 2 4 8 Guiding children in hands-on science activities 4.38 0 0 1 8 7 Asking children open-ended questions during hands-on science activities

4.25 0 0 3 6 7


4.44 0 0 2 5 9


4.50 0 0 2 4 10


3.50 2 1 4 5 4

N=13-16

16

24. How much of a challenge are the following when leading hands-on science activities with children in your classroom? (Circle one per row.)

Mean (1-5)

Not at All (1)

A L ittle

(2)

Somewhat

(3)

Very (4)

Extremely

(5) Classroom time constraints 2.93 1 3 7 4 – Classroom space constraints 2.81 1 5 7 2 1 Lack of necessary materials for activities 2.13 7 5 1 1 2 Lack of necessary equipment 2.25 4 8 1 2 1 Lack of necessary teacher resources 2.31 5 6 2 1 2 Needing to meet my center’s standards and requirements

2.13 6 5 2 3 0

Difficulty integrating supplemental science into our core curriculum

2.25 3 7 5 1 0

Difficulty getting children excited about science 1.81 9 4 1 1 1 My own comfort with science 2.19 5 5 4 2 0 Students’ language barriers 2.06 5 7 3 0 1 Students’ lack of interest in particular topics 2.31 2 10 2 1 1 Students’ discipline issues 3.00 0 6 6 2 2 Parents’ wishes for student outcomes 2.31 5 3 0 7 1 Other challenge 1.00 2 0 0 0 0

N=16

25. After completing the week-long Sid the Science Kid Summer Science Camp, in what ways, if any, have your views about science changed? (Check one for each row.)

Mean (1-3)

More than before

(1)

Less than before

(2)

About the same as before the week

(3) I’m comfortable doing science activities with kids in my classroom.

2.13 3 1 4

I’m confident in my ability to conduct science activities with kids in my classroom.

1.88 4 1 3

I’m confident in my ability to use television as an educational tool.

1.75 5 0 3

I’m interested in doing science activities with kids in my classroom.

2.00 4 0 4

I’m likely to incorporate science into our regular classroom routine.

2.00 4 0 4

I’m likely to use science vocabulary in conversations with kids in my classroom.

1.25 7 0 1

I’m likely to encourage kids to seek answers to their own questions

1.75 5 0 1

I’m likely to encourage kids to explore the world around them.

1.88 4 1 3

The range of activities I would consider “a science activity.”

1.00 8 0 0

N=8

26. How likely are you to do the following?

Mean (1-5)

Not at All (1)

(2)

(3)

(4)

Extremely (5)

Recommend viewing Sid the Science Kid to friends or colleagues. 4.88 0 0 0 1 7 Recommend attending a Sid the Science Kid workshop to friends or colleagues.

5.00 0 0 0 0 8

Recommend using the Sid the Science Kid curriculum to colleagues for use in their classrooms.

5.00 0 0 0 0 8

N=8 27. To what extent have the children in your class shown more interest in science learning? (Circle one).

Mean (1-5)

Not at all (1)

(2)

(3)

(4)

A great deal (5)

4.50 0 0 1 2 5 N=8

28. Please share a few specific examples (e.g., children asked to do more; comments about specific activities, etc).


29. What would you say the children in your class learned about science tools dur ing this unit?


30. What new things did you learn about science tools?


31. What additional information would you recommend including in the Sid the Science Kid cur r iculum mater ials to make the activities easier for teachers to use? Click here for a complete list of responses

Responses to “ Please explain which activities were most successful”

� Activities non-standard measurement were successful specially when they used their shoes, hands, body, blocks etc.

� As we worked on our rock candy activities, the children enjoyed taking part in measuring and viewing the sugar with magnifying glasses. They have been waiting and watching to see if any changes have occurred.

� Because it was new to the children and they want to do more than they were asked to do. They also have the chance to do some of the non-standard measurement at home. Some of them measured their bed and other measured their bed room.

� Because, children were curious about how long/big these sea creators were

18

� Children choose between red beans and lima beans. They had to place beans on wet towel paper and then into them on green plant bags-they observed and recorded changes vof the beans. Once the beans began to grow, they removed it from the bag into soil in a container. Kids measured the growth. They discussed which beans grew faster and which of the two beans did not grow.

� Children love using the magnifiers. They were excited to see the rocks, leaves, and sea shells in a different way with so much detail.

� The children loved using the magnifiers, rulers and creating charts. � The children really enjoyed exploring with the magnifiers. We have continued making charts to gather many

kinds of information. � The children respond with enthusiastic using the tools and they get the ability to use them by themself. Plus

they expand their vocabulary using the scientific words. � The children were able to predict what would happen to the sugar and watch it grow every day. � The children were very surprised when they saw how well things look through the magnifying glasses.

Sometimes we collect data on charts. Sometimes we collect it from each child individually and put everything together and make a book and share the information with the whole class.

� The month theme is farm animals. Some of the science questions discussed are, what kind of weather does a farm need to produce crops? What other types of farms are there?

� The science activity that children enjoyed was the rock candy. They really liked measuring and mixing the ingredients together. They liked the different colors that we used and have been anxiously waiting to see the rock candy be finished.

� They were able to see the process from beginning to end and observe change � We focused mostly on non-standard measurement, only briefly on linear measurement.

Return to annotated survey Responses to “ Which activities were the least successful?”

� The journals were too small for them to draw. The children are used to draw ib bigger paper. � It was too advance for the children � Making the connection between the topic and real life experience, it is not possible to visit a dairy

farm, ant farm, poultry, etc. � Comparing characteristic/similarities on things that we can find in the ocean � The children had difficulty drawing their observation. � The children found it harder to record their thoughts with pictures than to verbalize what they had

observed or explored. � Not that it wasn't successful, we just didn't focus as much on linear measurement � Journals were a little difficult because children wanted to draw pictures of their moms and dads

didn't quite get the concept � They experiment some difficult to draw charts � Children showed difficulties drawing what they see in the journal. � More abstract and difficult concept � Fossil making. After the activity was done, the children were not interesting in finding out about the

fossils. � Some concepts were a little difficult for them. � The children were able to measure the ingredients and mix them all together. Then the children

placed a dinosaur inside the mixture and wait for it to harden but it never got hard. � It was unsuccessful.

Return to annotated survey Responses to “How do you think your use of the Sid the Science Kid curr iculum mater ials would have been different if you didn’ t attend the training?”

� Because the kit was so easy to understand, I think it (the use of the kit) would have been the same. A lot of the material covered in the kit was already being utilized in the classroom in one form or the other

� I think the training is an excellent resource that helped me use the material successfully. For the children was also great. They learn better with tangible materials.

� I think the training was very useful. It would have been different if we weren't trained. I would have felt ill prepared and unfamiliar with the materials and concepts. Maybe 1 more day of training would be helpful.

� I was more comfortable using the materials because we had gone over them at the training. � I wouldn't describe the tools that is uses. For example, magnifying glasses, the program expanded the uses,

the charts. I use graph etc. � They were different because we had a very good explanation following the curriculum. � Yes. Very useful. It was key to make the camp excited and interesting and without the training I wouldn't

have any idea how to implement. Return to annotated survey Responses to “Descr ibe an example of how you used the video segments.”

� Both good � By episode using segments or each question or information. � I asked the children to pay attention to the question then I pause and have the children to predict

then we continue the video. � I did a brief introduction about the video first. Then I explain any vocabulary that was not familiar

for the children. After that we watched the video and then we implemented what we watched in the video

� I used them along with the videos and to reinforce other things in the classroom. � Stop the T.V. in any episode. Repeating the questions � To show the work that we would be doing � We introduced Sid first, talked about what we were going to view and then would play video, then

review what we saw. Return to annotated survey Responses to “What feedback, if any, did you get from parents regarding the family take-home mater ials?”

� During one weekend the teacher encounters one of her students on the street at his mother flower shop. The child was watching a Sid the Science Kid video that was sent home. He was very excited about. His mother said "I had to take the ruler. He was measuring all my plants.

� Parents were excited to hear their child's reactions at home. Children were excited about Sid. � Some parents gave me the comment that their children are talking about magnifying glass and the

tools like rule and journal. � The children shared their experiences with their parent. � The parents wanted that we send home more activities like the non-standard measurement. They

said that it was very good to see the children coming home asking them to help them to measure different object in their house.

� They said that the children are taking the magnifying glasses wherever they go. The parents have to explain to them and talk about things they observe with the magnifying glasses.

Return to annotated survey

20

Responses to “Please share a few specific examples (e.g., children asked to do more; comments about specific activities, etc.).”

� Children got very motivated. They wanted to know more about bugs. So we study the life cycle of a butterfly. One child found an ant on the floor. As soon as he said it, the rest of the children picked up the magnifying glasses and the exploration began.

� One of the kids tell me that they want to be measure with a block to see how long was her and she was counting the length of her body.

� The children repeated the activities during free choice time, weeks later. Measuring objects, making their own charts, etc.

� They are more interested in the use of microscope, magnifying glasses and rulers. � They enjoy the non-standard measurement � They loved the magnifiers � We were outside and one of the children saw a snail and he asked if he could go get his magnifying

glass to see it bigger. � When on a field trip, several children questioned each other on how to measure something they saw.

Children begin bringing in insects for each other to observe, both alive and dead. Return to annotated survey Responses to “What would you say the children in your class learned about science tools dur ing this unit?”

� Data, charts, graphing data � That a magnifying glass makes things look bigger. We use charts to record and share information. We use

rulers to measure. � The children learn that with non-standard measurement they only can use one item. � The names and what you can use them for and when you can use them. � They feel more comfortable using different materials; they develop their language and social skills. � They found a more concrete way to use them. � They learned new vocabulary. They also learned that with a magnifying glass things look bigger. They were

more aware about the size of things. The children also liked the idea of having their own journals to show and write.

� They learned the meaning of the word. Now they know that magnify make things look bigger, the chart is the same as graph and that needs to collect data

Return to annotated survey Responses to “What new things did you learn about science tools?”

� Children learn more. It's fun concepts are better understood, is easier for the children to make predictions and provide tangible evidence.

� Encourage children to create individual charts � I learn that non-standard measurement can be measure with anything but one item. � I learned a lot about non-standard measuring. � That you can use hands, feet, and other objects for non-standard measurement and only 1 item can be used at

a time. Return to annotated survey

Responses to “What additional information would you recommend including in the Sid the Science Kid cur r iculum mater ials to make the activities easier for teachers to use?”

� I loved it!!

� I would like to have the opportunity to have the material to teach with more advance time to get prepared. � Materials and information to explore internal human body. � Provide bigger science journals in they can compare their draws at the same time. � The materials given were perfect. The only thing I would suggest would be maybe for Sid to give out folders

to the children to make it more personal so they can associate the two. � The materials were great but again maybe several days can be given to each topic concept when doing the

weekly experiment. � The science journals should be bigger in size for preschoolers. They usually draw very big, so their drawings

didn't fit in the small pages.

Return to annotated survey Additional Comments

� For the purpose of research one week (5days) long is fine. Otherwise I would say to spread it out, and not make it consecutive. Only on Sid we have we used video inside the classroom.

� Two days for each would have given children better opportunity to explore each concept � We don't have TV, VCR, nor DVD player

22

Interviewer Play Activity Protocol

Child Name: _______________________

ACTIVITY OBSERVATION RECORD: TIME 2 Instructions for Interviewer: This activity record should be filled out as thoroughly as possible during the visit and then filled in online within 2 hours of the assessment. Use the back of the form if necessary. Date: ___________________ Time: ___________________ Researcher Initials: _______ DESCRIBE THE SETTING and where the materials have been arranged. Include important

details about the situation (e.g., # of children in the class, how close or far you are from the rest of the class, child is shy or cranky; child just woke from nap):

Who was in attendance during the play activity? ABOUT THE CHILD Age: �� 3 yrs old �� 4 yrs old �� 5 yrs old �� _____ yrs old Is the child a: � girl � boy? COMPLETE THIS SECTION AFTER THE PLAY ACTIVITY Total number of “activities” child engaged in: ____ # initiated by child: ____ # initiated by researcher: _____ # initiated by other _____ Who?: _____________________ Indicate ALL the materials child picked up and used in some way: � Ruler �� Popsicle sticks or blocks �� Magnifying glass �� Crayons

� Chart �� Stickers � Notebook

CHART (with or without stickers/with or without writing utensils)

Child did this activity: � First � Second � Third � Fourth � Fifth � Did Not Use Activity initiated by: � Child � Interviewer

� Other

Mark box if child does any of the following: (Check ALL that apply)

Uses the following terms:

� “compare” � “contrast” � “observe” � “information” � “chart”

� “data” � “science” � “scientist” � Other:

� “investigate” � “explore” � “discover”

Comments about the following in some way:

� Charts help us remember things

� Charts help us compare/contrast

� Chores; fruits; vegetables

Refers to a school lesson in some way: Describe:

� Asks you a question Specify:

Describe how the child used the tool, with what results, what he or she said. Child’s Verbal Observations (Put a “C” for child/ “I” for interviewer to indicate who initiated) Approximate length of time that child interacted with materials: (in minutes)

Please rate the child on the following dimensions. Consider the overall activity when determining the most appropriate rating.

The child was… Not at all

1 A little

2 Somewhat

3 Very

4 Extremely

5

At ease with the researcher 1 2 3 4 5

Talkative/verbal 1 2 3 4 5

Distracted 1 2 3 4 5

Interested in activity 1 2 3 4 5

Engaged in activity with “discovery” attitude 1 2 3 4 5

Methodical in approach to activity (seemed to have a plan) 1 2 3 4 5

24

MAGNIFYING GLASS

Child did this activity: � First � Second � Third � Fourth � Fifth � Did Not Use

Activity initiated by: � Child � Interviewer

� Other



� “compare” � “contrast” � “observe” � “magnify” � “tiny”

� “look bigger” � “science” � “scientist” � “tool”

� “investigate” � “explore” � “discover” � Other: Comments about the following in some way:

� Magnifying glasses help us see tiny things

� Magnifying glasses make things look bigger

� Rolie polies; bugs; leaves; shells






1 A little

2 Somewhat

3 Very

4 Extremely

5







MEASUREMENT



� Other



� “compare” � “contrast” � “observe” � “ruler” � “non-standard”

� “measure” � “science” � “scientist” � “tool”

� “investigate” � “explore” � “discover” � Other: Comments about the following in some way:

� Measuring things that are too big to measure easily

� Lining things up with no spaces in between

� Using other things to measure (besides a ruler)






1 A little

2 Somewhat

3 Very

4 Extremely

5







26

JOURNAL/NOTEBOOK



� Other



� “compare” � “contrast” � “observe” � “journal” � “draw”

� “keep track” � “science” � “scientist” � “organize”

� “investigate” � “explore” � “discover” � Other: Refers to a school lesson in some way:

Describe:





1 A little

2 Somewhat

3 Very

4 Extremely

5







OTHER ACTIVITY:



� Other



� “compare” � “contrast” � “observe” � Other: � Other:

� “keep track” � “science” � “scientist” � Other:

� “investigate” � “explore” � “discover” � Other:: Refers to a school lesson in some way:

Describe:





1 A little

2 Somewhat

3 Very

4 Extremely

5







28

Use the space below to jot down answers and a few DIRECT QUOTES if possible. � Did these objects remind you of anything that you’ve seen before? � Have you ever done anything like these activities we just did? � Do you know any scientists? � What kinds of things do scientists do? [Listen for and note whether child mentions: Science; Explore;

Discover; Investigations; Experiments; Observations; Scientists, etc] � Are you a scientist? � Is there anything else you’d like to talk about with me today?

Interview Questions: TIME 2 ONLY (3-5 min)

Onsite Classroom Observation Protocol

STSK Head Star t Observation Protocol Observation Date: __________________ Time Star t: __________ End: __________

Program: ___________________________ Leader: ___________________________

PART 1: DESCRIPTIVE INFORMATION ABOUT STUDENTS/TEACHERS Students: _____ # girls _____ # boys Student Race/Ethnicity: _____ # American Indian or Alaskan Native _____ # Asian _____ # Hispanic or Latino _____ # Black or African American _____ # Native Hawaiian or other Pacific Islander _____ # White _____ # Bi-racial/multi-racial _____ # Cannot be determined Teacher Gender: _____ Female _____ Male Teacher Race/Ethnicity: _____ American Indian or Alaskan Native _____ Asian _____ Hispanic or Latino _____ Black or African American _____ Native Hawaiian or other Pacific Islander _____ White

_____ Bi-racial/multi-racial _____ Cannot be determined

PART 2: DESCRIPTIVE INFORMATION ABOUT SESSION Day # ____: Activity/Title: _____________________________________________

Indicate how session time was spent. Activity Description Time Allotted to Activity Intro/Review/1st video clip __________ minutes Get up and move __________ minutes Intro/2nd video clip __________ minutes Intro/Journal __________ minutes Large group __________ minutes Small group __________ minutes Other __________ minutes Other __________ minutes Total __________ minutes

30

PART 3: IMPLEMENTATION How did the leader introduce/begin the session? When and how did the teacher distr ibute the mater ials? Consider ing only the time spent on the activities (excluding housekeeping activities, etc.), approximately what percent of this time was spent in each of the following arrangements? Whole group __________ % Pairs or small groups __________ % Individuals __________ % 100% Total Did the physical environment constrain the implementation of the activity? � Yes � No

If yes, please explain: Rate these indicators of implementation: Not at all

To a great extent

The teacher appeared prepared to lead the activity. 1 2 3 4 5 The materials for the activities were organized beforehand.

1 2 3 4 5

The teacher appeared confident in her ability to lead the activity.

1 2 3 4 5

The teacher appeared confident responding to children’s questions.

1 2 3 4 5

The teacher referred to the Binder. 1 2 3 4 5 The teacher’s activity management enhanced the quality of the activity.

1 2 3 4 5

The pace of the activity was appropriate for the developmental levels/needs of the children.

1 2 3 4 5

The teacher asked the children to reflect on their work. 1 2 3 4 5 The teacher’s questioning was likely to enhance learning (i.e., encourage students to reason about their observations and experiences, to draw relationships about what they have noticed, and to reflect on their work).

1 2 3 4 5

There was time for children to explore new or unfamiliar objects.

1 2 3 4 5

The leader engaged in exploration along with the youth. 1 2 3 4 5

PART 4: CHILD ENGAGEMENT Rate these indicators of youth engagement: Not at all

To a great extent

The children were engaged throughout the session. 1 2 3 4 5 The children used the materials successfully. 1 2 3 4 5 The children wrote in their Journals. The children asked questions of the leader. 1 2 3 4 5 The children reasoned about their observations and experiences, drew relationships about what they noticed, and/or reflected on their work.

1 2 3 4 5

The children discussed their ideas with each other. 1 2 3 4 5 The children had fun doing the activities. 1 2 3 4 5 Were there any objects used dur ing the session that seemed new or unfamiliar to youth? � Yes � No

If yes, how did the youth react? Synthesis Statement Brief description of the nature and quality of youth engagement. Include examples and quotes.

Sid the Science Kid Summative Evaluation · G O O D M A N R E S E A R C H G R O U P , I N C . M a r...

Documents

Transcript of Sid the Science Kid Summative Evaluation · G O O D M A N R E S E A R C H G R O U P , I N C . M a r...