Statistics Netherlands and Big Data › dokument › 8685 › Statistics-Netherlands...Big Data Team...
Transcript of Statistics Netherlands and Big Data › dokument › 8685 › Statistics-Netherlands...Big Data Team...
“Learn from details, act global”
Statistics Netherlands and Big Data
Marco Puts
Big Data Team
2
Martijn Tennekes
Piet Daas
Joep Burger
Marco Puts
From Survey to Big Data
Primairy data Secondary data
Our ‘own’ surveys Data from ‘others’ - Administrative sources
- Big Data
Statistics Netherlands
4
Big Data: Another Perspective
The internet
had 1800
exabytes of
data in 2011
exa = 10^18
NOW IDC/EMC white paper 2008
CES 2011 Geneva
5
Data Deluge: 50.000 EXAbytes in 2020
We live in exponential times
27 fold growth in the next 9 years
Stategic Aspects of Big Data
Statistics at Statistics Netherlands
• Known and good quality
• Slow
Statistics at commercial institutes
• Unknown Quality
• Fast
Bigger urge for Fast and Reliable Statistics
Data deluge
Attention for positioning Statistics
Netherlands w.r.t. Big Data
6
Big Data Organization: the Roadmap
Strategic importance of Big Data
Responsibility at DG level
Roadmap Big Data: o Mobile phone data
o Traffic loop data
o AIS
o Social Cohesion
Half-yearly update of Roadmap
Coordination Group Big Data
7
Isues
Used Methods
Stability of Sources
Representativity
Privacy
Usability
Infrastructure
Knowledge and Skills
Unknown or nonmatching Metadata
Quality of big Data
8
Big Data Processes Data driven vs. output driven
9
Stats Big Mess of Data
Quality of Big Data The Signal and the Data
10
Manually
Automatic
Q Q
Process parameters
Data Signal
Noise
Big Data Team Backgrounds
11
Martijn Tennekes
Piet Daas
Joep Burger
Marco Puts
Biology Bio-informatics
Computer Science Cognitive Science Psychophysics
Biology
Knowledge engineering
Secret of our Success
Experimental Skills
Explorative Research
Artificial Intelligence
Computer science
12
Some Data sources and Projects
13
Sentiment on social media o “Consumer Confidence”
o Safety indicator
Road Sensor Data o Traffic intensities
AIS data (Automatic Identification System for Vessels) o ESSnet
o Maritime traffic around the Netherlands
o Inland waterway traffic
o Emissions of vessels
Mobile phone meta data o Daytime population
o Tourism
Social Cohesion o Relationship between people on social media
The Future
The future
of statistics
looks
BIG 14