Playful Explorations of Public and Personal Data - OSCON Data 2011

Post on 19-Jan-2015

2.314 views 1 download

Tags:

description

It’s easy to find and create data. But what are you going to do with it? Can I ask the world complex questions such as what’s the local crime rate, distance to metro, or rating of my local school? Can you combine these all together to rate houses you may want to buy? And how do you then connect back to your government and local businesses to engage in collaborative decision making.This talk with discuss how you should consider users and their personal interactions with data and information. We’ll also peel back the covers on how open source tools such as HBase, Cascading, Geos and Polymaps handle analyzing and streaming realtime data to maps and visualizations both on the web and to mobile devices.To illustrate what’s possible, we’ll dive through GeoCommons, a large online community of data sharing and community analytics that uses open source mapping visualization, Hadoop analysis, and mobile interfaces to provide this to the world. Users can even build and socialize their own analysis methods to share their expert knowledge with other users. We’ll also review how global organizations like the World Bank and United Nations are using these tools to connect with citizens in developing countries to empower them to make decisions on building investment and understanding how climate science may affect their areas.

Transcript of Playful Explorations of Public and Personal Data - OSCON Data 2011

Pub

lic &

Personal

DataPlayful

Explorations

Andrew Turner@ajturner

Data

Open Government Data

flickr: jrparis

of the world80%live within cell coverage

flickr: jrparis

of the world80%live within cell coverage

and of the50%world own a mobile phone

flickr: jrparis

unread POI’s 1000+

Black Friday, 2010

Social Media Trajectorieshttp://geocommons.com/maps/89371#

Personal Data

How to deal with this data? flickr: ejpphoto

Approachable, relevant, accessible

Sharing Visualizations http://flowingdata.com

Tools are often too difficult.

Basic Principles

Open Access

Basic Principles

Open Access

Visualize

Basic Principles

Open Access

Visualize

Analyze

Basic Principles

Power of Place

The role of mapsMaps Reflect & Define Reality

Waldseemüller Uniersalis Cosmographia

The role of mapsMaps Reflect & Define Reality“America”

Waldseemüller Uniersalis Cosmographia

Story Mapping

Navigation Marshall Islands stick chart19th century

Travel Guides

Spatial Operations

Buffer, Intersect, AggregationCorrelation, Prediction, Custom

Tools: geos, JTS, PostGIS

Spa$al  Aggrega$on:  Group  data  by  arbitrary  geographic  boundaries

(Proprietary  and  confiden0al)

Spa$al  Aggrega$on:  Group  data  by  arbitrary  geographic  boundaries

(Proprietary  and  confiden0al)

Spa$al  Aggrega$on:  Group  data  by  arbitrary  geographic  boundaries

(Proprietary  and  confiden0al)

Spa$al  Correla$on:  Rela0onship  between  mul0ple  datasets  by  geography

(Proprietary  and  confiden0al)

Spa$al  Correla$on:  Rela0onship  between  mul0ple  datasets  by  geography

(Proprietary  and  confiden0al)

Spa$al  Correla$on:  Rela0onship  between  mul0ple  datasets  by  geography

(Proprietary  and  confiden0al)

Spa$al  Buffer  and  Distance  Filter:  Radius  buffer  from  a  point,  line  or  polygon

(Proprietary  and  confiden0al)

Spa$al  Buffer  and  Distance  Filter:  Radius  buffer  from  a  point,  line  or  polygon

(Proprietary  and  confiden0al)

Spa$al  Buffer  and  Distance  Filter:  Radius  buffer  from  a  point,  line  or  polygon

(Proprietary  and  confiden0al)

Intersec$on:  Overlap  between  two  polygonal  areas,  lines  or  points

(Proprietary  and  confiden0al)

Intersec$on:  Overlap  between  two  polygonal  areas,  lines  or  points

(Proprietary  and  confiden0al)

Intersec$on:  Overlap  between  two  polygonal  areas,  lines  or  points

(Proprietary  and  confiden0al)

Intersec$on:  Overlap  between  two  polygonal  areas,  lines  or  points

(Proprietary  and  confiden0al)

Intersec$on:  Overlap  between  two  polygonal  areas,  lines  or  points

Use  Cases:• Locate  that  a  social  media  men0ons  happened  within  a  geo-­‐fence

• Track  mobile  traffic  entering  a  zip  code

(Proprietary  and  confiden0al)

User-­‐Defined  Analysis:  Custom  equa0ons  via  intui0ve  wizard  interface

(Proprietary  and  confiden0al)

User-­‐Defined  Analysis:  Custom  equa0ons  via  intui0ve  wizard  interface

(Proprietary  and  confiden0al)

User-­‐Defined  Analysis:  Custom  equa0ons  via  intui0ve  wizard  interface

(Proprietary  and  confiden0al)

Traffic Accidents http://geocommons.com/maps/77512

Virtual Vehicle www.virtualvehicle.co

Crash ‘Likelihood’ Density www.virtualvehicle.co

Making it revelantflickr: qwertyuiop

neogeography |nē'ō-jē-ŏg'rə-fē|

geographical techniques and tools used for personal activities or for utilization by a non-expert group of users;

Psychogeography

Playful Explorations

http://geocommons.com

http://geocommons.com

67k+ unique datasetsbillions of features.

590,000+ geographic attributes

Acetate Layers

http://developer.geoiq.com

= 174,049,705,000

= 174,049,705,000

well... close enough

Open Access

Open Access

Visualize

Open Access

Visualize

Analyze

Citizens

Foursquare Crime

Citizens

Government

Social Government

Citizens

Government

Media

Journalists

Journalists

Citizens

Science

Media

Government

Research

Citizens

Science

MediaDevelopers

Government

Using Appcelerator’s open-source cross-mobile platform technology, mobile applications can quickly integrate with GeoIQ to access and create new data from the field.

GeoIQ & Appcelerator

GeoPace

HBase

GEOS

GeoPacing

GeoPace

Action

Results

Search

Loc(lat,lon)

HBase

GEOSstart

GeoPacing

GeoPace

Action

Results

Search

Loc(lat,lon)

HBase

GEOSstart

GeoPacing

GeoPace

Action

Results

Search

Loc(lat,lon)

HBase

GEOSstart

Cascading

GeoPacing

GeoPace

Action

Results

Search

Loc(lat,lon)

HBaseDB

DB

DB

GEOSstart

Cascading

GeoPacing

GeoPace

Action

Results

Search

Loc(lat,lon)

HBaseDB

DB

DB

GEOSstart

Cascading

MapMapMapMapMapMapMap

GeoPacing

Appcelerator

http://appcelerator.geoiq.com

Mobile Analysis on Mobile Devices

Citizens

Organizations

Science

MediaDevelopers

Enterprise

Social Media Dashboards MongoDBPolymaps

Organizations

Science

MediaDevelopers

Enterprise

Citizens

Hotness http://geocommons.com/maps/89561

Health http://geocommons.com/maps/77282

Traffic & Caffeine http://geocommons.com/maps/82402

More Technical Information

Open Geospatial Libraries for RubyChris HelmWednesday @ 5PM

Andrew Turner - @ajturnerandrew@geoiq.comhttp://highearthorbit.com

Thank You