Polly

Polly: Viral Entertainment as a Vehicle for Disseminating Core Development ServicesAgha Ali Raza, Roni Rosenfeld

Language Technologies InstituteCarnegie Mellon UniversityPittsburgh, PA, USAFarhan Ul Haq, Zain Tariq, Mansoor Pervaiz, Samia Razaq, Umar Saif

School of Science and EngineeringLahore University of Management SciencesLahore, Pakistan

1Our TeamIntroGoalsPollyAnalysisPlans

Agha Ali Raza Roni Rosenfeld Farhan Ul Haq Zain Tariq Mansoor Pervaiz Samia Razaq Umar Saif

Our Long Term VisionSpeech-based information access for low-literate people in developing countries

Examples of information services:speech-based Craig's Listspeech-based citizen journalismspeech-based message boards/blogshealth, agriculture, jobs, education,IntroGoalsPollyAnalysisPlansWe want to facilitate development among low-literate usersWe want to do that by providing them access to information and communication technologiesAnd we believe that it can be effectively done using speech based services over simple phones

We envision information services like Speech based access to:health informationagricultural informationJob search

And communication services like, speech-based:Market placeMessage boards and blogsCitizen journalismThe Case for SLT4DPCs are not feasible for low-literate massesNon-affluent cannot afford themUnskilled cannot operate themStable electricity, Internet etc. cannot be relied uponLow cost phones may be a feasible solution

Low-cost phones often only support voice & SMSVideo not feasibleGraphical interactions not always possible

Text is problematic for low literate usersSpeech may be the only viable interaction paradigmIntroGoalsPollyAnalysisPlansHow can we reach the low-literate?Speech over simple phones is a viable way to reachlow-literate people in developing countries

IntroGoalsPollyAnalysisPlans

Non-affluent cannot afford themUnskilled cannot operate themStable electricity, Internet etc. cannot be relied upon

Modes of Input- Speech-based input

- Push button (DTMF) inputIntroGoalsPollyAnalysisPlansProblem 1: User TrainingEven simple IVR systems are often a challenge for the low-literate

Speech Interfacesusually require user training e.g. Healthline (Sherwani et al 2009), Avaaj Otalo (Patel et al 2010)

Explicit training is not a scalable solution

Solution: Incentivize people to train themselvesWhere there's a will there's a way [Smyth et al 2010]

Entertainment turns UI Barriers into mere speed bumps

IntroGoalsPollyAnalysisPlans

7Problem 2: How to Mass Disseminate?How to advertise / communicate to masses who dont read?

Solution: Viral Spread

IntroGoalsPollyAnalysisPlans

8Goal of this workDevelop Viral Entertainment as a vehicle for disseminating Development related telephone based services

Introduce and popularize speech interfaces Use Entertainment as a Viral Conduit for delivering core development servicesSetup an Experimental Testbed for testing speech interface choicesProvide Entertainment

IntroGoalsPollyAnalysisPlansOur First Attempt: SonglineA telephone-based, voice-based application which allows users to listen to songs recorded by others, as well as to record their own songs and to forward them to friends

Spoken prompts in Urdu and English for outputDTMF (push button) for inputAirtime paid by us (missed call mechanism)Developed using Tropo speech and telephony platform (tropo.com)

IntroGoalsPollyAnalysisPlans10Discuss verbally onlywww.tropo.comSongline: User InterfaceListen to recorded SongsRecord a Song- Browse songs by most popular or most recent Sing after a beep Listen to song and confirm Enter phone number(s) of friend(s) Record an introductory message

IntroGoalsPollyAnalysisPlans11Songline User Study FeedbackAbout the idea:Songs and music are considered controversial

Privacy concerns

About the Interface:Entering phone numbers is tedious. Users prefer selecting from phone directory

Busy tone confused users

Detailed call tree and numerous options confused users

IntroGoalsPollyAnalysisPlans12Polly ( )Polly is a telephone-based, voice-based application which allows users to make a short recording of their voice, modify it and send the modified version to friends.

IntroGoalsPollyAnalysisPlans

13IntroGoalsPollyAnalysisPlans

14Pre-PilotPilot tested in Lahore in early March 2011Seeded with two office workers at LUMSTest lasted two weeksUser base increased to 32We then stopped Polly and gathered feedback by interviewing the usersSoftware flaws detected and fixed

IntroGoalsPollyAnalysisPlans2011 - Pilot LaunchSingle incoming phone lineLaunched on March 31, 2011Seeded with the 32 usersRemained up for 22 days Amassed 2,032 users who took part in 10,629 interactions

Reasons for Shut down:Line capacity saturated within two weeksLine remained busy during peak hours resulting in frustrated users The international call charges ($0.126/minute) were becoming a significant financial burden for usIntroGoalsPollyAnalysisPlans

2012 Large Scale DeploymentDeployed locally in LahoreCall charges went down to $0.023/minuteIncreased capacity to 30 phone linesIncreased voice manipulations to sixAdded Job Audio-BrowserIntroGoalsPollyAnalysisPlans

First Information Service: Job Audio-BrowserScan Pakistani newspapers for jobs for low-skilled workers

Record these adsInvite Pollys users to audio-browse them

IntroGoalsPollyAnalysisPlans

Ref: paperpk.com182012 Large Scale DeploymentLaunched on May 09, 2012:Gave the phone number to 5 low-literate peopleAfter 141 days (mid-September 2012):495,000 calls85,000 usersSpreading to 1,000 new people daily

27,000 people used the job search servicelistened 279,000 times to job ads and forwarded them 22,000 times to their friends.IntroGoalsPollyAnalysisPlans

2012 Large Scale DeploymentIntroGoalsPollyAnalysisPlans

2012 Large Scale Deployment

IntroGoalsPollyAnalysisPlans

Enthusiasm is Lost Quickly:IntroGoalsPollyAnalysisPlans

But a few people continue long term

22User Demographics

Determined by listening to a sample of recordings: IntroGoalsPollyAnalysisPlans

undetermined (young children, old people, too much background noise, etc.).5388 recordings23User DemographicsDetermined from 207 survey callsIntroGoalsPollyAnalysisPlans

Low SES(No Education)Socio-Economic Status24

Randomized Controlled TrialsVery convenient RCT setup!Hundreds of new users join every day

So far we have been paying all airtime charges using missed call mechanismGoal: Reduce our airtime charges while maintaining system spread among the poorQuestion: How will various quotas affect user behavior?

Is Polly compelling enough for people to spend their own money on it, at least sometimes?

IntroGoalsPollyAnalysisPlans

Randomized Controlled TrialsIntroduced a caller-paid lineCalls picked up, caller pays airtimeImpose various quotas (Qx) on the number of toll-free calls (x) per day (for each user). When quota exceeded, direct caller to caller-paid lineQ7Q3Q2Q1IntroGoalsPollyAnalysisPlans

RCT: Effect of Daily Quota of 7 CallsIntroGoalsPollyAnalysisPlans

Caller-paid LineIntroGoalsPollyAnalysisPlans

2012-13 Large Scale DeploymentLaunched on May 09, 2012 in Lahore, Pakistan:Seeded with 5 low-literate peopleAfter 1 year:636,000 calls165,000 usersAt its peak it was spreading to 1,000 new people daily

34,000 people used the job search servicelistened 386,199 times to 728 job ads and 19,000 users forwarded them 34,000 times to their friends.IntroGoalsPollyAnalysisPlans

Geographical Spread

IntroGoalsPollyAnalysisPlans

And also a handful of calls from:IndiaBelgiumOmanSaudi ArabiaUAE

Current WorkRe-launch Polly in Pakistan to serve as a clearinghouse for jobs and skill-training opportunities

Make Polly viral in India with better mechanisms of populating Job Ads and tracking employment outcomes

Multi-platform support

Much more analysis to be doneDetailed logs, audio recordings,...Collaborators welcome!

IntroGoalsPollyAnalysisPlans

Can such a system scale up to millions of calls? Yes!Core development-related service: Job audio-browsingGood uptake (many started calling specifically for this service!)Can it be made cost effective?No conclusive answer yet:audio ads, content sponsors, carrier revenue sharingPolly as an experimental testbed: Used mostly by uneducated young men (some mid- and high-SES)Interest declines within ~4 days, for most*Most users very sensitive to airtime costLarge dataset of social interaction, currently being analyzedLet us know if you want to work on it!Thank you!

Contact us: www.cs.cmu.edu/~PollySummaryPayload Options

Job SearchEducationEntertainmentAgricultureCommerce???Health careOur Strategy:33Distributed Setup for Pilots

web1234TropoCMULUMS

IntroGoalsPollyAnalysisPlansUser Feedback

Period of feedback recordings listened to:Aug 4Sep 9Total number of feedback recordings listened to: 2567Of those, containing thanks or praise to Polly 1797 (70%) containing other feedback 272 (10%)Of those containing other feedback: System-initiated 183 (67%) User-initiated 89 (33%) Complaints about: 189 (70%) Delay in call-back time 140 Poor sound quality 28 Failure of system to send/receive message 14 Being disturbed by Polly 5 Requests/suggestions: 83 (30%) Higher quota or more recording time 47 New services: News, Weather, Medical, Voice Chat, Job Ads, Songs. 22 More voice manipulations 8 No voice manipulation 6 Envisioned Payload ServicesDisseminate speech-based communication services for low-literate telephone users throughout the developing world Buying and selling goods and services (i.e. a speech-based equivalent of Craig's List, craigslist.com) Finding and communicating with others who share a common interestFacilitating social and political activism (speech-based message boards)Expressing opinions and making them broadly accessible to others (speech-based blogging and tweeting)Sending and receiving group messages (speech-based mailing lists)Broadcasting and receiving announcements in emergenciesCitizen journalismJob Search, Education and Remote Training

IntroGoalsPollyAnalysisPlansOur First Attempt: SonglineA telephone-based, voice-based application which allows users to listen to songs recorded by others, as well as to record their own songs and to forward them to friends

Spoken prompts in Urdu and English for outputDTMF (push button) for inputAirtime paid by us (missed call mechanism)Developed using Tropo speech and telephony platform (tropo.com)

IntroGoalsPollyAnalysisPlans37Discuss verbally onlywww.tropo.comSongline: User InterfaceListen to recorded SongsRecord a Song- Browse songs by most popular or most recent Sing after a beep Listen to song and confirm Enter phone number(s) of friend(s) Record an introductory message

IntroGoalsPollyAnalysisPlans38Songline User Study FeedbackAbout the idea:Songs and music are considered controversial

Privacy concerns

About the Interface:Entering phone numbers is tedious. Users prefer selecting from phone directory

Busy tone confused users

Detailed call tree and numerous options confused users

IntroGoalsPollyAnalysisPlans39Voice EffectsOriginal Voice: Professor Roni Rosenfeld

An I-have-to-run-to-the-bathroom effect

2) A drunk chipmunk effect

3) Converting the voice to a whisper

4) Adding background music

5) The original, unmodified recording

IntroGoalsPollyAnalysisPlans

40Subsidy Transition

Subsidized calling costs: We pay for all the airtime of Polly calls (missed call mechanism).Partially unsubsidized calling costs: user bears the cost of calling Polly; we bear the cost of delivering the message to friends.Completely unsubsidized calling costs: users bear the cost of calling Polly as well as of receiving the message (maybe Polly gives a missed call or sends an SMS to the intended recipient).IntroGoalsPollyAnalysisPlansWhere we work

IntroGoalsPollyAnalysisPlans

http://placeswallpaper.blogspot.com/2010/06/badshahi-mosque.html422012 Large Scale DeploymentIntroGoalsPollyAnalysisPlans