Teachers' Use of Classroom Connectivity Technology: Log File Analysis

Vehbi Aytekin Sanalan

sanalan.1@osu.edu

3/7/2008 SITE 2008 2

Project team

Principal Investigators Douglas T. Owens, OSU Stephen L. Pape, UF Karen E. Irving, OSU Louis Abrahamson, BEF Frank Demana, OSU

Post doctoral researcher Vehbi Sanalan, OSU &

Erzincan University, Turkey

Consultants Christy Kim Boscardin, UCLA Joan Herman, UCLA Jeremy Roschelle, SRI

Project Coordinator Mike Kositzke, OSU

GRAs Sedat Ucar; Sukru Kaya;

Melissa Shirley; Clare Bell; Ugur Baslanti; Sharilyn Owens; Hyesook Shin; Gonul Sakiz; Jonathan Bostic

3/7/2008 SITE 2008 3

CCMS Project Overview

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305K050045 to The Ohio State University.  The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education.

Interdisciplinary professional development and research project Algebra I and Physical Science achievement Student self regulated learning skills Teaching strategy change in formative assessment

practices

Summer Institute – training T3 conference follow-up (2 years) Classroom connectivity technology

3/7/2008 SITE 2008 4

Classroom Connectivity Technology

Create a collaborative learning environment

Engage in formative assessment by way of immediate feedback

Enhance classroom management of TI graphing technology

Quick Poll provides teacher understanding by receiving impromptu feedback

The TI-Navigator System allows the teacher to:

3/7/2008 SITE 2008 5

Technology Components

Quick Poll

Screen Capture

Activity Center

Class Analysis/Learning Check

3/7/2008 SITE 2008 6

Implementation Fidelity

Control/Treatment

Group assignment is dichotomous

Implementation is usually not uniform

Data collection for implementation

Teacher reports

Observations

Software tracing

Issues with each procedure

3/7/2008 SITE 2008 7

Log File Literature

Analysis of user tracing documents can provide information about actual use of technology (Saye & Brush, 2001; Nurmela, 1999; Reips & Stieger, 2004)

It fails to provide detailed understanding of educational settings (Boving & Simonsen, 2004; Nyvang & Bygholm, 2005; Owston, 2000)

Standard logging mechanism Timestamp of the event Type of event Source of event

3/7/2008 SITE 2008 8

Log File Structure

Participant TeacherTime and means of submission

Class session Class name Starting and ending time of the class Number of students logged in

Component type Starting & ending times Number of Ss enrolled Quick Poll - question probe Class Analysis - test name Activity Center - type of info exchanged Screen Capture

3/7/2008 SITE 2008 9

Log File Structure

Class and activities schemeSchool allocated time for class

session (X=59’55, sd. 25’29)

Class session duration from log files

(X=52’38, sd. 13’58)

Average QuickPoll usage (X=2’58, sd. 14’26)

Average Activity Center Usage (X=4’31, sd. 7’58)

Average ClassAnalysis usage (X=44’41, sd. 39’56)

3/7/2008 SITE 2008 10

Log File Data Preparation

Preparing raw data for analysis

Date and time limits

Number of students

Excessive class duration

Activities without students

3/7/2008 SITE 2008 11

Phone Interview Data Rating

Teachers were asked to; estimate the number of times they used the

technology per week

rate their use of each component (never,

sometimes, often and every class session)

Team of researchers used a coding scheme

to establish the inter-rater reliability and

finalized the rate for each teacher.

3/7/2008 SITE 2008 12

Congruence

Average weekly use information from

different sources are correlated (R=.466, p≤.001)

but significantly different (t44=3.43, p ≤.001)

Component use information is also correlated

for Class Analysis and Screen Capture (R=.620

and .380)

3/7/2008 SITE 2008 13

Results - Participants

Out of 98 total participants 50 teachers from 28 (21) different states

Gender Female: 75% (76%) Male:25% (24%)

Ethnicity White 87% (92%) African-American 9% (4%) Hispanic 4% (4%)

First year vs. second year users 49% (50%) No significant difference between subgroups on

average use of technology

3/7/2008 SITE 2008 14

Results – Total Use

Average use per week N Percentage (%)

less than once 10 20

1-2 times 18 36

3-4 times 9 18

More than 5 times 13 26

3/7/2008 SITE 2008 15

Results – Component Use

Screen capture31%

Quick poll38%

Activity center16%

Class analysis15%

3/7/2008 SITE 2008 16

Results – Student Participation

Component Percent

Overall 91%

Quick poll 85%

Activity center 93%

Class analysis 97%

3/7/2008 SITE 2008 17

Results – Tech Use Over a Year

Total compomnent use by months

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Aug-06 Sep-06 Oct-06 Nov-06 Dec-06 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07

Screen Capture

Quick Poll

Activity center

Class analysis

3/7/2008 SITE 2008 18

Next Steps

Analysis of content and log files in

conjunction with classroom observations

Include log file analysis in the implementation

fidelity index derived from telephone

interviews

HLM analysis using log file data set

Thank you very much!

For electronic copy of this presentation:

http://www.ccms.osu.edu or ask sanalan.1@osu.edu

(Please allow a couple of days until it shows up on the web)