Data Visualization Daniel Silver March, 2014 [number of slides courtesy of Stan Matwin]

Data Visualization

Daniel SilverMarch, 2014

[number of slides courtesy of Stan Matwin]

The KDD Process

Selection and Preprocessing

Data Mining

Interpretation and Evaluation

Data Consolidation

Knowledge

p(x)=0.02

DataWarehouse

Data Sources

Patterns & Models

Prepared Data

ConsolidatedData

The KDD Process

The Architecture of a KDD System

Graphical User Interface

DataConsolidation

Selectionand

Preprocessing

DataMining

Interpretation, Visualization

and Evaluation

Warehouse KnowledgeData Sources

Big Data, Saint John, NB 4

Data Visualization

• The greatest value of a picture is when it forces us to notice what we never expected to see – J. Tukey

• Visual analytics can lead to discoveries that neither a computer nor a human could make alone – J. Fekete

• Let your data talk to you – J. Stasko


What is visualization?

• Graphical representation of data and relationships within data

• Why?– right (imagery) and left (analytical) brain

hemisphere• What makes a good visualization?– Informative – Aesthetically pleasing– Often, the right abstraction of the data


Windmap

• http://hint.fm/wind/ [Martin Wattenberg]– Captures the “Big Picture” from the data– Data from the National Digital Forecast Database– Exhibited in MOMA as graphical art

http://hint.fm/wind/


Visualization as a tool for…

• Data exploration• Communication of results


How best to visualize?

Art is good in conveying

complex concepts


Advertising uses great visualizations…


But is it that easy …

• … art conveys emotions, mental states…

• Visualizing data is different, but still…– A good visualization should require no explanation– Allows us to see the data “in a new light”


Geographic misconceptions…

• The True Size of Africa [Kai Kruse]• http://flowingdata.com/2010/10/18/true-size-

of-africa/

http://flowingdata.com/2010/10/18/true-size-of-africa/

http://flowingdata.com/2010/10/18/true-size-of-africa/


Conveying Complex Dynamics

• Projecting on the time axis– Using colour, thickness, etc. to combine and

overlay data


[Minard 1896!]


Email “mountain” [F. Viergas] -making sense of an email archive


Visual exploration process [Yau, Data Points, 2013]


Exploring data

• ACOA database of 28,000 projects• Contains – Client’s address, postal code– Amount of funding– Project description– Program type– ….


ACOA data

Project Number Client Name Client Address Client City Client Postal Code Project Description Project Location Project Geographic Code Program Type Assistance Type ACOA Assistance

197595 (ISIS) Immigrant Settlement and Integration Services Society Halifax, B3L 4P1 Halifax B3L 4P1 IBDS funding for 2010 - 2011 HALIFAX 1209034

Business Development Program

Non-Repayable Contribution 149,832.00

198632 (ISIS) Immigrant Settlement and Integration Services Society Halifax, B3L 4P1 Halifax B3L 4P1Immigrant entrepreneur and business succession integration

ATLANTIC REGION 1400000

BDP - Atlantic Population Strategy


200029 (ISIS) Immigrant Settlement and Integration Services Society Halifax, B3L 4P1 Halifax B3L 4P1The Immigrant Women Entrepreneurial Project (IWEP)

ATLANTIC REGION 1400000

BDP - Atlantic Population Strategy


157116 * Quidi Vidi Rennies River Development FoundationSt. John's, A1B 2Z2 St. John's A1B 2Z2 Environmental Education Expansion ST. JOHN'S 1001519 Partnerships


154136 * Burin Peninsula Community Business Development Corporation Inc.Marystown, A0K 2M0 Marystown A0K 2M0 implement a youth business program MARYSTOWN 1002024

ACOA - Action Program (AAP) Grant 6,800.00

154135 * Burin Peninsula Community Business Development Corporation Inc.Marystown, A0K 2M0 Marystown A0K 2M0 Youth Internship Program - Phase 3 MARYSTOWN 1002024 Community Futures


113504 042691 N. B. LtéePetit Rocher Nord, E8J 2K7 Petit Rocher Nord E8J 2K7

To acquire equipment & make changes to the ventilation,water PETIT-ROCHER 1315014

ACOA - Action Program (AAP)

Unconditionally Repayable Contribution 9,520.00

185785 047100 N.B. Inc.Bouctouche, E4S 2J2 Bouctouche E4S 2J2 Lean Analysis BOUCTOUCHE 1308005

AIP - ISDI - SME Technology Advancement


203111 051996 N.B. Ltd.Mundleville, E4W 2N5 Mundleville E4W 2N5

Building expansion and purchase of equipment for winery RICHIBUCTO 1308018



167460 052412 N-B LtéeTracadie-Sheila, E1X 1G6 Tracadie-Sheila E1X 1G6

Expansion - pool, washrooms, electrical, road upgrades.... SAUMAREZ 1315001



167466 052412 N-B LtéeTracadie-Sheila, E1X 1G6 Tracadie-Sheila E1X 1G6

Chalet/Campground expansion - adding activities SAUMAREZ 1315001



183896 053247 N.B. LtéeSt-Quentin, E8A 2H9 St-Quentin E8A 2H9 Construction of potato storage SAINT-QUENTIN 1314022



168364 054000 NB Inc.Beresford, E0B 1H0 Beresford E0B 1H0 Purchase 2 blueberry harvestors BERESFORD 1315015 Agriculture

Non-Repayable Contribution 182

169969 056393 N.B. Ltd.Bathurst, E2A 4Z2 Bathurst E2A 4Z2 Estab. of Beach Cleaning Service BATHURST 1315011

ACOA - Action Program (AAP)



Geo mapping

• We map the Postal code of each ACOA project to geographic coordinates using the PCCF Canada Post database


ACOA Data [Tableau]


ACOA Program Types

• Business Development Program

• Productivity and Business Skills (PBS)

• ACOA Action Program (AAP)

• Other


ACOA data – geolocated by project type


Visualization and text

WordcloudwithR


Beyond wordcloud

• How to characterize at a high level what the project is all about?

• Find, by analyzing all project descriptions, high level “clusters” (“topics”) of descriptions can be identified


5 LDA topics for ACOA projectsTopic 1 Topic 2 Topic 3 Topic 4 Topic 5

conference equipment equipment activities program

development development activities existing development

existing plan plan equipment business

facility facility business new system

equipment business host host new

system expansion centre development equipment

plan operation expansion training market

study new new operation study

activities activities market business tourism

project centre project support international

event equipment-add equipment-new training-support study+tourism


So where is the future? - challenges

• Visualization of stream/dynamic data (animation?)

• Visualization of very large graphs• Visualization of multi-dimensional data (see

OECD life quality demo)• Visualization for mobile devices (limited

screen sizes, limited power)• Evaluation of visualization

Data Visualization Daniel Silver March, 2014 [number of slides courtesy of Stan Matwin]

Documents

Transcript of Data Visualization Daniel Silver March, 2014 [number of slides courtesy of Stan Matwin]