NYPL Internet Lending pre-pilot OST survey results

20
NYPL Mobile Hotspot Lending 100 Unit Test Preliminary Metrics July 2014

description

Internet Lending 100 unit pilot survey results.

Transcript of NYPL Internet Lending pre-pilot OST survey results

Page 1: NYPL Internet Lending pre-pilot OST survey results

NYPL Mobile Hotspot Lending100 Unit Test

Preliminary MetricsJuly 2014

Page 2: NYPL Internet Lending pre-pilot OST survey results

2

We have developed a logic model to identify how we can measure impact against the programs 4 goals.

Goal 1Increase students/household access to at home broadband internet

Goal 2Raise digital exposure and confidence

Goal 3Increase engagement with library-provided and/or recommended learning resources

Goal 4Establish a scalable public library model for tackling the national digital divide

Page 3: NYPL Internet Lending pre-pilot OST survey results

3

TALLY DATA

Page 4: NYPL Internet Lending pre-pilot OST survey results

• A total of 106 households showed interest in the program by completing a user interest form

• Of those who communicated interest, 84% were eligible

• A total of 91 households are participating in the pilot (98% of those eligible)

• A total of 91 wireless routers and 50 Chromebooks were loaned Chromebooks were only made available to Out of School Time (OST) students

• 34 households checked out both devices; 47 checked out only the wireless router; and 6 checked out only the Chromebook

• 47 participants are ESOL students at Bronx Library Center (BLC); the remaining participants are parents of OST students at Mott Haven (19), Highbridge (13), and Port Richmond (12)

Lent

0 5 10 15 20 25 30 35 40 45 50

Wifi & ChromebookWiFiChrome Book

Spanish

English

4

Goal 1Increase students/household access to at home broadband internet

NOTE: • 37% of participating households primarily speak

Spanish• TechConnect not channel not tested

Page 5: NYPL Internet Lending pre-pilot OST survey results

5

ISP DATA

Page 6: NYPL Internet Lending pre-pilot OST survey results

6

There are three program goals measured through ISP and survey data

Goal 1Increase student/household access to at home broadband internet

Goal 2Raise digital exposure and confidence

Goal 3Increase engagement with library-provided and/or recommended learning resources

Page 7: NYPL Internet Lending pre-pilot OST survey results

Activity measurement limitations and related assumptions

7

Method: Activity measured by connections on a particular port.We are not given full destination website addresses due to privacy regulations of the ISP. We were provided the port labels of the destination networks. While not perfect, it does provide a hint as to the nature of activity because certain activity (such as email, streaming video, social media, gaming, chat and messaging) which occurs on uniquely labeled ports.

Assumption: Chromebook ActivityThe port traffic label SPDY we assume to be Google Chromebook traffic. Because we lent Chromebooks to 50% of our participants that traffic may potentially skew the results but only by 2%. “Google” activity for this analysis is essentially Chromebook/Google Apps activity.

Assumption: Anonymous TrafficWe assume that the "Anonymous" traffic (traffic on ports commonly labeled "HTTP") consists of Google traffic as well. Google owns roughly 68% of all web traffic so we assume 68% of the "Anonymous" traffic is Google traffic as well.

NOTE: For the 10K pilot, if we chooses Sprint, we will be able to route all traffic through a monitoring service to intercept CIPA restricted sites (black listed web site categories.) We will therefore, have greater visibility as to the category of traffic based on the full address. This will be anonymous and in aggregate so we preserve privacy.

Page 8: NYPL Internet Lending pre-pilot OST survey results

Users are spending an average of 3.19 hours online per day, and most often at times when the library is closed (6PM – 12AM local)

12:00 AM

2:00 AM

4:00 AM

6:00 AM

8:00 AM

10:00 AM

12:00 PM

2:00 PM

4:00 PM

6:00 PM

8:00 PM

10:00 PM0

500000

1000000

1500000

2000000

2500000

3000000

3500000

Time of Day (UTC)

Conn

ectio

ns

• Average Time on-line: 3.19 hours/day

• This equates to 191 minutes vs a single 40 minute session at library during opening hours

8

Goal 1Increase students/household access to at home broadband internet

Note: TIME of Day on chart is UTC so subtract 3 hours for local time.

Page 9: NYPL Internet Lending pre-pilot OST survey results

9

YouT

ube

Video

App S

tore

Mus

ic

Onlin

e File

Sto

rage

Phot

o sh

arin

g

Inst

ant M

essa

ging

VOIP

Comm

erce

News

Map

s

Onlin

e App

s

Utility

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

% Activity vs % Data Volume

% Traffic % VolumeData

384.7 GB/mo173.2 GB/mo102.7 GB/mo51.3 GB/mo

HH Usage GB/mo

4.23 GB/HH/mo1.90 GB/HH/mo1.13 GB/HH/mo0.56 GB/HH/mo

Activity

Anonymous YouTubeEncryptedVideo

Goal 1Increase students/household access to at home broadband internet

Goal 2Raise digital exposure and confidence

We see healthy traffic levels around >9GB per household per month, most of which is anonymous

Page 10: NYPL Internet Lending pre-pilot OST survey results

File Sharing, Social Media, YouTube, and Google Web Properties seem to represent bulk of traceable activity online.

Anonymous68%

File Sharing14%

En-crypted6%

So-cial Media6%

YouTube3%

Google2%

Entertainment1%

95% of Traffic

AnonymousFile SharingEncryptedSocial MediaYouTubeGoogleEntertainment

Anonymous48%

YouTube22%

Encrypted13%

Video

6%

So-cial Media4%

App Store3%

Entertainment1% Music

1%

95% of Volume

AnonymousYouTubeEncryptedVideoSocial MediaApp StoreEntertainmentMusic

10

Goal 2Raise digital exposure and confidence

Page 11: NYPL Internet Lending pre-pilot OST survey results

11

Usage of educational content is not measurable through ISP data for the 100 unit test, so we will rely on exit surveys to measure this goal

Anonymous48%

YouTube22%

Encrypted13%

Video6%

Social Media4%

App Store3%

Entertainment1%

Music1%

95% of Volume

AnonymousYouTubeEncryptedVideoSocial MediaApp StoreEntertainmentMusic

For the 10K test, we will have the ability to capture URL traffic programmatically.

We could assume Khan Academy is part of the YouTube activity for our OST students. Other resources like Mango for ESOL or Math Whiz or NYPL are most likely masked due to port labeling and lack of URL logging.

Goal 3Increase engagement with library-provided and/or recommended learning resources

Page 12: NYPL Internet Lending pre-pilot OST survey results

12

SURVEY DATA – PRELIMINARY

Survey data includes 23 responses from parentswith children participating in the library’s OST program.

Page 13: NYPL Internet Lending pre-pilot OST survey results

Participants report an average of 2.2 adults and 1.6 children per household, or an average of 3.8 total household users

1 2 30%

10%

20%

30%

40%

50%

60%52%

35%

13%

Child Users (ages 5+) per Household

# of child users

% o

f house

hold

s

1 2 3 4 50%

10%

20%

30%

40%

50%

60%

22%

57%

9% 9%4%

Adult Users per Household

# of adult users

% o

f house

hold

s

13

Goal 1Increase students/household access to at home broadband internet

preliminary survey n = 23

Page 14: NYPL Internet Lending pre-pilot OST survey results

$50K - 75K

$25K - 50K

Under $25K

0% 20% 40% 60%

11%

50%

39%

Total Household Income

% of households

Employed full-time

Employed part-time

Not employed, looking for work

Student

0% 20% 40% 60%

27%

55%

14%

5%

Employment Status – Head of Household

% of households

14

Goal 1Increase students/household access to at home broadband internet

preliminary survey n = 23

Nearly all households report incomes under $50K, and less than one third of all heads of household are employed full-time

Page 15: NYPL Internet Lending pre-pilot OST survey results

15

The majority of respondents report that they have gained confidence in both their own and their children’s ability to use the Internet

Own Ability (Adult)

Child's Ability0%

10%20%30%40%50%60%70%80%90%

100%

26% 19%

43% 57%

30% 24%

Much more confident than before

More confident than before

About as confident as before

Since participating in the Internet Lending pilothow confident are you in your own / your child’s ability to use the

Internet?

Goal 2Raise digital exposure and confidence

preliminary survey n = 23

**“Much less confident” and “less confident” were also answer options

Page 16: NYPL Internet Lending pre-pilot OST survey results

Finding or applying for a job

Connecting with friends / family (e.g. Skype, Facebook)

Learning job-related skills

Keeping informed of current events

Checking information about child's school

Helping child with schoolwork

0% 20% 40% 60% 80% 100%

39%

39%

43%

57%

70%

91%

% of respondents

Nearly all respondents used the Internet to help their children with schoolwork, and many used it for job-related purposes

In the past two months, have you used the Internet

for any of the following reasons?

16

Goal 2Raise digital exposure and confidence

preliminary survey n = 23

Page 17: NYPL Internet Lending pre-pilot OST survey results

If offered Internet service for $10 / month, the majority of respondents feel it is somewhat or very likely they would purchase it in the next six months

How likely is it that you will purchase a computer or Internet service in the next six months?

0%10%20%30%40%50%60%70%80%90%

100%

30%10%

13%

19%

13%

13%

38%

30% 33%Very likely

Somewhat likely

Neither likely nor unlikely

Somewhat unlikely

Very unlikely

17

Goal 2Raise digital exposure and confidence

preliminary survey n = 23

Page 18: NYPL Internet Lending pre-pilot OST survey results

18

The vast majority of respondents were promoters of both the library’s wireless Internet and Chromebook services

90%

10%

NPS – Internet Service

PromoterDetrac-torPassive

Net Promoter Score = 80

96%

4%

NPS – Chromebook

PromoterDetrac-torPassive

Net Promoter Score = 96

Goal 2Raise digital exposure and confidence

preliminary survey n = 23

Net Promoter measures the likelihood of patrons recommending our services to a friend or coworker by asking, “on a scale of 0 to 10 (10 being the highest and 0 being the lowest), how likely are you to recommend (X) to a friend or colleague?” Promoters are people who select 9 or 10, passives 8 or 7, and detractors those who picked 0 through 6. The Net promoter score is then calculated by subtracting the % of promoters by the % of detractors.

Page 19: NYPL Internet Lending pre-pilot OST survey results

Nearly all households intend to renew their devices

19

Goal 2Raise digital exposure and confidence

preliminary survey n = 23

How likely is it that you will renew the deviceafter the current loan period ends?

Wireless Router

Chromebook0%

10%20%30%40%50%60%70%80%90%

100%

5% 5%5%5% 10%

85% 86%

Very likely

Somewhat likely

Neither likely nor unlikely

Somewhat un-likely

Very unlikely

Page 20: NYPL Internet Lending pre-pilot OST survey results

Khan Academy

Math Whiz Power My Learning

0%

20%

40%

60%

80%

100%

18% 14% 23%9% 9%

9%

27% 32%

41%

36% 32%14%

9% 14% 14%

In the past two months, how often did your child visit the following websites?

Daily or more

A few times a week

Once a week

20

Goal 3Increase engagement with library-provided and/or recommended learning resources

preliminary survey n = 23

Respondents indicated frequent usage of all three library-recommended learning resources included in the survey