HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining...
Transcript of HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining...
![Page 1: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/1.jpg)
HunchLab: Spatial-Temporal Data Mining Robert Cheetham
10 April 2009 Workshop on Spatial-Temporal Modeling
![Page 2: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/2.jpg)
Geographic
Analysis
Geoprocessing
Web Services &
Solutions
Custom Web
Apps
![Page 3: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/3.jpg)
![Page 4: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/4.jpg)
![Page 5: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/5.jpg)
![Page 6: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/6.jpg)
![Page 7: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/7.jpg)
PhillyHistory.org
![Page 8: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/8.jpg)
![Page 9: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/9.jpg)
![Page 10: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/10.jpg)
Crime Mapping
![Page 11: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/11.jpg)
Connect 211
![Page 12: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/12.jpg)
Connect 211
![Page 13: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/13.jpg)
![Page 14: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/14.jpg)
![Page 15: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/15.jpg)
![Page 16: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/16.jpg)
Weighted Raster Overlay
x 5 x 2x 3x 1
+ ++
=
![Page 17: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/17.jpg)
AfricaMap
![Page 18: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/18.jpg)
AfricaMap
![Page 19: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/19.jpg)
Background
![Page 20: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/20.jpg)
1,500,000 humans
7,000 police
1,000 civilian employees
2,000,000 new incidents / year
Philadelphia Context
3 crime analysts
![Page 21: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/21.jpg)
Weekly Compstat
Lots of maps
Automation of map creation
Web-based systems
What we did
![Page 22: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/22.jpg)
- accelerate the cycle
- proactively notify
- automatically run
… But what if we could
![Page 23: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/23.jpg)
1999 Internal research project at Phila PD
2000 Paper for NIJ Crime Mapping Conference
2004-2005 Prototype
2007-2010 National Science Foundation R & D Grant
Crime Early Warning System
![Page 24: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/24.jpg)
ArcViewVB & MapObjects
MS SQL Server
Crime Incidents
Database
Shapefiles
and
GRIDs
Process
Documentation
.ini
file
Philadelphia Context
![Page 25: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/25.jpg)
Philadelphia Context
![Page 26: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/26.jpg)
Philadelphia Context
![Page 27: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/27.jpg)
Philadelphia Context
![Page 28: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/28.jpg)
Bad News
It was crap…
…sort of
![Page 29: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/29.jpg)
We needed
1. Better Statistics
2. Notification
3. Super-simple
![Page 30: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/30.jpg)
Round 2: Crime Spike Detector
![Page 31: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/31.jpg)
Round 2: Crime Spike Detector
Burglaries, 6/20/2004, 1275ft. Radius, 30 Day Period Each Year
![Page 32: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/32.jpg)
Round 2: What we learned
1. People used it but not the way we expected
2. More flexibility
3. More customized to user needs
![Page 33: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/33.jpg)
� Automatic detection
� Subscribe to search patterns
� Automatically e-mail
� Configurable
� Fast
� Centralized
� Individualized
� Applicable to any space-time event
� No new software or hardware
Round 3: HunchLab Objectives
![Page 34: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/34.jpg)
What is HunchLab?
HunchLab DatabaseCrime
Database
GIS Database
Alerts
Alerting SystemGeostatistical Engine
Crime Mapping Tools
Web Map Service
HunchLab Web Site
![Page 35: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/35.jpg)
A crime spike is an “above average” amount, but:
How do we know when to send an alert?
� How much above average?
� What “average” are we using?
� When? Over what period of time?
� Where?
� What category of event?
![Page 36: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/36.jpg)
1. Location
2. Time
3. Type of Event
Basic Data Requirements
![Page 37: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/37.jpg)
1. Location
2. Time
3. Filter
Setting up a Hunch
![Page 38: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/38.jpg)
Four options:
1) Address + Radius;
2) District/Sector/Beat
3) Custom Polygon
4) Mass Hunch
Hunch Parameters – Location
![Page 39: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/39.jpg)
Time: two parts:
1) Current Data (recent past)
2) Historical Data
Hunch Parameters – Time
![Page 40: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/40.jpg)
Hunch Parameters – Time
![Page 41: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/41.jpg)
Event Filter:
� Category
� Time of Day
� Text (future)
Hunch Parameters – Event Filter
![Page 42: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/42.jpg)
Hunch Parameters – Type of Hunch
� Statistical Hunch
� Event Trigger
![Page 43: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/43.jpg)
Notification
![Page 44: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/44.jpg)
Email Notification
![Page 45: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/45.jpg)
![Page 46: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/46.jpg)
![Page 47: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/47.jpg)
![Page 48: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/48.jpg)
� Too large area or too long time � nothing is a spike.
� Too small area or too short time � everything is a spike.
� Different crime classes require different calibration.
Defining Hunches can be tricky because…
![Page 49: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/49.jpg)
� Doesn’t need lots of detailed data …
� … but has to be accurate data
� Humans still required
� No explanatory or interpretive power
� Not good for low volume events
The small print ….
![Page 50: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/50.jpg)
High volume
databases:
humanly
impossible to track
all the data
When and where
has something
different
happened?
Geographic
Notification
![Page 51: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/51.jpg)
Large space-time events databases
� Public Health
� Real Estate Sales
� Other data sets
Not just for criminals anymore….
![Page 52: HunchLab: Spatial-Temporal Data Mining - Harvard University...HunchLab: Spatial-Temporal Data Mining Robert Cheetham 10 April 2009 Workshop on Spatial-Temporal Modeling. Geographic](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edcf992ad6a402d6667e33b/html5/thumbnails/52.jpg)