Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

ANALYTICS AND OPEN DATA THROUGH A CASE STUDY

SAS MIDDLE EAST

CAREL BADENHORSTHEAD OF INFORMATION TECHNOLOGY PRACTICE

MIDDLE EAST

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

SAS AGENDA

• Analytics and Open Data

• Analytics example - UN Global Pulse Case Study

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

Discover relevant themes and relationships in social media, call

notes and email for deeper insights and improved business

management

Understand and find relationships in data to make accurate predictions about the future

Leveraging historical time series data to drive better insight into decision-makingfor the future

Make appropriate business decisions by

understanding dynamics and utilize

resources the best way

FORECASTING

DATA MINING

TEXT ANALYTICS

OPTIMIZATION

STATISTICS

INFORMATIONMANAGEMENT

Copyright © 2011, SAS Institute Inc. All rights reserved.

ANALYTICS LIFE CYCLE….NOT BI

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

SAS/ UN GLOBAL PULSE

BACKGROUND OF THE CASE STUDY

The UN Global Pulse- SAS research had a

few questions

• Does the sum total of what we say online

add up to anything meaningful?

• Do online conversations correlate in any

way with official government statistics?

• Specifically can unemployment patterns be

predicted based on certain chatter topics

and correlated with govt open data to derive

meaningful statistics?

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

SAS/ UN GLOBAL PULSE

METHODOLOGY

Online social media conversations over a

period in US and Ireland

Government Open Data to validate experiment ie.

Employment history statistics

Mood Scoring based on conversations

Text Analytics - words used in each conversation were mined in order to assign one or more

topical categories

Sentiment Analysis undertaken to classify

conversations as happy, sad, anxious etc

Dynamic Correlation between mood scores with

unemployment scores

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

SAS SAMPLE INSIGHT GENERATED FROM THE RESEARCH

RESULTS

• An uptake in social media conversation on topics such as cutting back on groceries and other essentials or downgrading one’s mode of transportation can predict an impending unemployment spike. 

• After a spike, an increase in chatter about foreclosures, reduced spending for health care and canceled vacations can offer insights on the effects of a down economy. 

• Better understanding of demographical areas, gender, age and income characteristics based on social techniques such as mood scoring

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

SAS SAMPLE INSIGHT GENERATED FROM THE RESEARCH• In the US: • Huge increase in depressed mood conversations four months before a spike in

unemployment (calculated and validated within 95 percent). • Talk about loss of housing increases two months after an unemployment spike (calculated

and validated within 95 percent). • Talk about auto repossession increases three months after an unemployment spike

(calculated and validated within 95 percent).

• In Ireland: • Anxious moods increase five months before a spike in unemployment (calculated and

validated within 90 percent)• Talk about travel cancellations increases three months after an unemployment spike

(calculated and validated within 95 percent). • Talk about changing housing situations for the worse increases eight months after

unemployment increases (calculated and validated within 90 percent).

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.

SAS TRANSLATING INTO• In the US:

• 95 Confidence EARLY WARNING SIGN KPI (four months) for unemployment increase• 95 Confidence EARLY WARNING SIGN KPI (six months – four plus two months) for

mortgage repayment default increase (down to the demographics)• 95 Confidence EARLY WARNING SIGN KPI (seven months – four plus three months)

for car manufacturers and retail re new sales and potential default increase (down to the

demographics)• Increased potential in social welfare needs down to a specific demographic level……

and the most important value

• Using further analytics statistical, data mining, prediction and optimization algorithms to

start predicting pre-emptive actions and their outcome in case these patterns are

detected• Analytics is amazing if you allow it to tell you stories....

Copy r ight © 2013, SAS Ins t i tu te Inc . A l l r ights reserved.sas.com

THE STORIES ANALYTICS WILL HELP YOU TELL USING OPEN DATA IS ENDLESS…

QUESTIONS?