A Preliminary Profile of Student Tablet Use During the Lecture

Presented by Nicole Chittenden, Assistant Dean, School of Business

Administrationhttp://www.bsad.uvm.edu/

June 12, 2007Southeast Regional Conference 2007

(SERCOMP)Atlanta, GA

Based on research partially funded by Microsoft and conducted by Associate

Professor James Kraushaar, Ph.D., Assistant Professor David Novak, Ph.D.

and Thomas Chittenden, Lecturer© James Kraushaar et al., 2007. This work is the intellectual property of the authors.

Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given

that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors. 1

AbstractBSAD @ UVM has required a student-purchased

tablet PC since Fall 04 [pen-enabled for Fall 07] Faculty suspect some students might be using

the technology in ways that detract from the learning process

Some feel strongly enough to ban laptops from their lectures while others have encouraged them

Understanding how laptops/tablets impact the process requires knowing what students are doing with their PCs during the lecture

2

AgendaBSAD @ UVM – History/who are

we?HypothesisRecorded activity dataPreliminary findings/resultsMobile Computing Usage SurveyLessons learnedFuture stepsAudience feedback?

3

BSAD @ UVM – History/who are weAACSB International accreditation for UG & MBA30 FT Faculty (5 MIS)13 staff (2.5 IT Administrators)

IT Manager/Systems Engineer; Developer [Web Master/SQL DB/Programmer]

Student workers = Work studies & part-time wages = ~ 1 FTE

Will add ¾ time IT support technician in 07/081000 UG [25% ^ in last 2 years], 70 MBA, 150 Minors (UVM

~10,000)2004 moved from laptop to Tablet PC requirement

FA07 # of tablets = ~800 = ~80% studentsUVM Support = On-site reseller, service

technicians, replacement parts & loaners

4

Why a Computer Requirement?Required by BusinessesCost shifting to student Students drive the introduction of new technologyTablets are consistent with goal of customized

educational experience & have flexibility of notebook computers and more!

Students aren’t as prepared as we think – “Results from ETS indicate generally poor performance

on the organization’s new test, the ICT Literacy Assessment Core Level” 13% of the 3,000 college and 800 high school students were deemed computer literate.”*

* Educause, 11/12/2006 Tech Watch, p. 4 “ETS announces results of Information Literacy Test” quoted from Chronicle of Higher Education October 2006 http://educause.edu/ir/library/pdf/erm0664.pdf

5

Tablet historyAll have had Onboard CDRW/DVD drive, 4-Year Parts & Service & 4-Year Accidental Damage

Warranties – Fall 2007 added 4-year Battery Warranty

6

2007…Gateway® E-155C Lighter, more portable

1.17-inches thin12.1” touch screen

durablelightweight (~5 lbs)

biometric fingerprint reader ultra low voltage

2006-2007Gateway®

M285-EBigger, heavier,

extra horsepower 14.0” widescreen

2004-2005Gateway®

M275Very light

Slim14.0”

PROJECT: HypothesisSome instructors feel very strongly that students

are easily distracted by technology and laptops facilitate goofing around in the classroom and create an environment where students are physically present, but mentally absent. Students are bringing a full entertainment center

into the classroom (TV, DVD, communication, etc)On the other hand, there are instructors who feel

very strongly that technology usage in the classroom is critical in explaining certain topics/subjects and that laptops are tools that modern day students must become comfortable with using in a productive manner.

7

Curriculum Redesign – Junior/Senior Course:Management Information Systems

Leveraged Tablet Functionality for All Diagrammatic Concepts in ClassEntity Relationship DiagramsData Flow DiagramsSystems Development Life Cycles

Homework = required MS Journal & electronic submission

Implemented PC Skills Assessment at beginning & endActivity monitoring software tracked in class usage of

PCRe-Vamped Lectures to Leverage Tablet PC as

chalkboard replacement8

Recorded Information Methodology99 of 229 juniors/seniors, ~600 Hours of

recorded student activity, 7 classes, 2 instructors

Signed Consent forms & Activity Monitor Software installed on student machines.

Students initialized the recording application and terminated it in each class

Activity ‘harvested’ at the end of classWe put spyware on student machines!!

Problems with Recorded Activity DataStudent owned & managed machines = little control

over devicesVirus/Malware protection repeatedly eliminated

softwareStudents forgot to start recording applicationStudents shut machine down before harvest routineLabor intensive process for instructor to manage both

the coordinated recording process & the collection process

Varying recorded times between studentsAbsenteeism, inability to manage software…18 hours recorded for one student vs. 15 minutes

recorded for some students.Bottom line: convenient sample…not a random sample

11

Relationships between academic performance and recorded activityLooking at 2 courses with the most activity

monitor data…% of time instant messaging has a significant

negative correlation (-.476)Only ‘Productive Activity’ category had a

positive correlation (.391) with statistical significance to overall academic performance.

When looking at just Tablet Users, the correlation increases to +.65

Tablets change the mix of preferred note taking applications

50% of browsing activity is “unproductive”…

50% of web browsing done in class is unproductive!

The most compelling data supporting the notion that Tablet PC Functionality enabled better retention of diagrammatic concepts was found in the grade performance in two sections of the observed course.

Somewhat fluctuating….compared to Females, this doesn’t show much of a difference between Tablet owners and Laptop owners.

Females with Tablets performed far higher than females with laptops

Paired T-Test (Tablets vs. Laptops) Tablet Users

Laptop Users

Tablets & Laptops

Note: 124 of 229=60% of possible participants

Meann=58

Meann=66

MeanN=124

Start of Semester Technology Readiness Assessment Score 78.97 82.09 80.63

End of Semester Technology Readiness Assessment Score 85.47 86.97 86.27

Difference +6.50 +4.88 +5.64

Tablet users showed larger improvement in a standardized digital literacy assessment

Mobile Computing Usage SurveyJunior & senior level students ~40% tablet80 Questions about Student computer usage

At homeIn all their classesIn their Information Systems Classes

192 students out of 229 completed (83%)Not Anonymous

Are Tablets a more embraceable computing solution?

Tablets are more likely to be brought to class

Tablets are more likely to be ‘used’ during class

Tablet Users were 15% more likely to think PC’s are NOT a Distraction in Class

Laptops are perceived as more reliable than Tablets

Tablet owners are not as satisfied with their devices as Laptop owners

Tablets have a lower perceived value

Students don’t appreciate ‘requirements’

Technology in our CurriculumWe are Committed to Technology in

CurriculumSatisfaction: 6 for 6 green arrows in 2005 &

2006 re: IT training & teaching per annual Educational Benchmarking Institute (EBI) graduating exit survey

Performance: Educational Testing Services (ETS) Major field test in Business = Quant/MIS rank in top 95% over the last 2 academic years

Dare we Conclude: Students are satisfied with IT & it is a measurable strength?

30

Lessons LearnedOn average, students use distracting

software for 20% of class timeTablets facilitate learningTablet PC Owners are less satisfied with

overall value, reliability & system/hardwareActivity Monitor data

Is cumbersome to administer , maintain & collectCan be misleading (active application)

31

Future StepsRegarding Computer Requirement:

Entwine Computing Usage survey with a National standardized test in Business

Evaluate other monitoring solutions for class management (DyKnow, Altiris…)

Re-visit lease program = controlled systems!

Consider teaching in controlled lab environment

Regarding Research:

Consider visual observation data as opposed to Soft Activity

Currently teaching Summer course in a controlled lab

Improve experimental design

32

Thanks to:This presentation is based on research

partially funded by a 2-year Microsoft Tablet PC grant

Microsoft Research Grant Investigators:James Kraushaar, Ph.D., Associate Professor, School of Business

Administration, UVMJeff Frolik, Ph.D., Assistant Professor, School of Engineering, UVM

David Novak, Ph.D., Assistant Professor, School of Business Administration, UVM

Thomas Chittenden, MBA, Lecturer, School of Business Administration, UVM

33

How to contact us:

http://www.bsad.uvm.edu james.kraushaar@uvm.edu

thomas.chittenden@uvm.edu david.novak@uvm.edujeff.frolik@uvm.edu

nicole.chittenden@uvm.edu

34