Post on 21-Jan-2018
Peter Aiken PhD
Why Johnny Can't Data
DATA
and what you can do about it
Peter Aiken, Ph.D.
Copyright 2016 by Data Blueprint Slide #
• 30+ years in data management • Repeated international recognition • Founder, Data Blueprint (datablueprint.com) • Associate Professor of IS (vcu.edu) • DAMA International (dama.org) • 9 books and dozens of articles • Experienced w/ 500+ data management practices • Multi-year immersions:
– US DoD (DISA/Army/Marines/DLA) – Nokia – Deutsche Bank – Wells Fargo – Walmart – …
• DAMA International President 2009-2013
• DAMA International Achievement Award 2001 (with Dr. E. F. "Ted" Codd
• DAMA International Community Award 2005
PETER AIKEN WITH JUANITA BILLINGSFOREWORD BY JOHN BOTTEGA
MONETIZINGDATA MANAGEMENT
Unlocking the Value in Your Organization’sMost Important Asset.
The Case for theChief Data OfficerRecasting the C-Suite to LeverageYour Most Valuable Asset
Peter Aiken andMichael Gorman
2
50 Years of IS at VCU
Copyright 2016 by Data Blueprint Slide # 3
Copyright 2016 by Data Blueprint Slide # 4
DATA
Copyright 2016 by Data Blueprint Slide # 7
Why Johnny Can't Data?• If your are satisfied with your current
data leverage capabilities then … – Sit this talk out – Data is a by-product of IT operations – Detritus (debris, waste, refuse, rubbish, litter, scrap, flotsam and jetsam, rubble)
• IT Challenge – Lack of foundational data content/incorrect focus on technologies – Data is a resource – Growing awareness that it needs to be governed
• Knowledge Worker Challenge – The epitomy of knowledge work – Data is an asset – Most powerful, yet most underutilized and poorly managed asset
• What next? – A challenge to your organization
US DoD Reverse Engineering Program Manager
Copyright 2016 by Data Blueprint Slide # 8
• "Your first project is to keep me from having to testify to a Congressional Hearing!"
• Problem: 37 systems paid personnel within DoD
– How many were needed?
– How many potential losers?
• What do you mean by employee?
• Process modeling - inconclusive results
• Data reverse engineering - definitive
– One legged engineer, working in waist deep waters, underneath rotating helicopter blades, on overtime
Data Reverse Engineering
Amazon Best Sellers Rank: #1,841,642 in Books
Copyright 2016 by Data Blueprint Slide # 9
UntanglingThe Legacy knot
Copyright 2016 by Data Blueprint Slide # 10
You can accomplish Advanced Data Practices without becoming proficient in the Foundational Data Practices however this will: • Take longer • Cost more • Deliver less • Present
greaterrisk(with thanks to Tom DeMarco)
Data Management Practices Hierarchy
Copyright 2016 by Data Blueprint Slide #
Advanced Data
Practices • MDM • Mining • Big Data • Analytics • Warehousing • SOA
Foundational Data Practices
Data Platform/Architecture
Data Governance Data Quality
Data Operations
Data Management StrategyTechnologies
Capabilities
11
Governor's Data Interns Program
Copyright 2016 by Data Blueprint Slide # 12
Commonwealth Agencies • VDOT • DARS • ELECT • DMAS • DHRM • SecHealth • DMV • VDH • …
Screening
Two Phase Approach
Copyright 2016 by Data Blueprint Slide # 13
Packaging(INFO609)
ResultsINFO611/632
} Opportunities for
Results
Fast Track MBA
Other INFO Classes
Other University Participants
One concept for process improvement, others include:
• Norton Stage Theory
• TQM
• TQdM
• TDQM
• ISO 9000
and focus on understanding current processes and determining where to make improvements.
DMM℠ Capability Maturity Model Levels
Copyright 2016 by Data Blueprint Slide #
Our DM practices are informal and ad hoc, dependent upon "heroes" and heroic efforts
Performed (1)
Managed (2)
Our DM practices are defined and documented processes performed at the
business unit level
Our DM efforts remain aligned with business strategy using standardized and consistently implemented practices Defined
(3)
Measured (4)
We manage our data as a asset using advantageous data governance practices/structures
Optimized
(5)DM is strategic organizational capability, most
importantly we have a process for improving our DM capabilities
14
Copyright 2016 by Data Blueprint Slide # 15
Organizations Surveyed• Results from more
than 500 organizations
• 32% government
• Appropriate public company representation
• Enough data to demonstrate that European organizational DM practices are generally more mature
Copyright 2016 by Data Blueprint Slide #
Data Management Strategy
Data Governance
Data Platform & Architecture
Data Quality
Data Operations
0 1 2 3 4 5Client Industry Competition All Respondents
Challenge
Challenge
Challenge
16
Comparative Assessment Results
Data Management Strategy
Data Governance
Data Platform & Architecture
Data Quality
Data Operations
Industry Specific Results
• CMU's Software Engineering Institute (SEI) Collaboration
• Results from hundreds organizations in various industries including: – Public Companies – State Government Agencies – Federal Government – International Organizations
• Defined industry standard • Steps toward defining data management "state of
the practice"
Copyright 2016 by Data Blueprint Slide # 17
Focus: Implementation and
Access
Focus: Guidance and
Facilitation
Optimizing (V)
Managed (IV)
Documented (III)
Repeatable (II)
Initial (I)
DM Maturity 2007-2011
Copyright 2016 by Data Blueprint Slide # 18
1
2
3
4
5
Data
Man
agem
ent S
trate
gy
Data
Gov
erna
nce
Data
Plat
form
& A
rchit
ectu
re
Data
Qua
lity
Data
Ope
ratio
ns
2007 Maturity Levels 2011 Maturity Levels
Copyright 2016 by Data Blueprint Slide # 19
What do they mean big?"Every 2 days we create as much information as we did up to 2003"
– Eric Schmidt
Copyright 2016 by Data Blueprint Slide # 20
The number of things that can produce data is rapidly growing (smart phones for example)
IP traffic will quadruple by 2015 – Asigra 2012
The likely state of your data management efforts
Copyright 2016 by Data Blueprint Slide # 21
Very
Silo
’ed
or co
nflic
ting
data
sour
ces
Multiple Data Sources Inconsistent
data definitions
of common
terms
ISD are data
ownersLots of
Data….Minimum
Informatio
n Inconsistent Data Quality
Difficult to report and mine against
Redundancy Multiple changes to source system
There will never be less data
than right now!Copyright 2016 by Data Blueprint Slide # 22
DATA
Copyright 2016 by Data Blueprint Slide # 23
Why Johnny Can't Data?• If your are satisfied with your current
data leverage capabilities then … – Sit this talk out – Data is a by-product of IT operations – Detritus (debris, waste, refuse, rubbish, litter, scrap, flotsam and jetsam, rubble)
• IT Challenge – Lack of foundational data content/incorrect focus on technologies – Data is a resource – Growing awareness that it needs to be governed
• Knowledge Worker Challenge – The epitomy of knowledge work – Data is an asset – Most powerful, yet most underutilized and poorly managed asset
• What next? – A challenge to your organization
Copyright 2016 by Data Blueprint Slide # 24
When I grow up ... Pole dancing home depot shovel sales
Dear Ms. Davis: I want to be very clear on my child's illustration. It is NOT of me on a dance pole on a stage in a strip joint. I work at Target and had commented to my daughter how much money we made in the recent snowstorm. This drawing is of me selling a shovel. Mrs. Harrington
Copyright 2016 by Data Blueprint Slide # 25
Target Isn't Just Predicting Pregnancies
Copyright 2016 by Data Blueprint Slide #
http://rmportal.performedia.com/node/1373 and http://www.predictiveanalyticsworld.com/patimes/target-really-predict-teens-pregnancy-inside-story/ http://rmportal.performedia.com/rm/paw10/gallery_01#1373
26
Managing Data with Guidance?
Copyright 2016 by Data Blueprint Slide # 27
Lewis in front of the cummins safe
Copyright 2016 by Data Blueprint Slide # 28
And in this corner we have Dave!
Copyright 2016 by Data Blueprint Slide # 29
John McClain Bruce Willis crawling thru air ducts
Copyright 2016 by Data Blueprint Slide # 30
A high-level graphic showing the various routes that Verizon penetration testers were able to use to get all the way down to Target’s cash registers in 2013 and 2014
Copyright 2016 by Data Blueprint Slide # 31
http://krebsonsecurity.com/2015/09/inside-target-corp-days-after-2013-breach/
Copyright 2016 by Data Blueprint Slide # 32
Beth Jacobs abruptly resigned in March
These decisions have consequences!
Copyright 2016 by Data Blueprint Slide #
and my point is …
33
DATA
Copyright 2016 by Data Blueprint Slide # 34
Why Johnny Can't Data?• If your are satisfied with your current
data leverage capabilities then … – Sit this talk out – Data is a by-product of IT operations – Detritus (debris, waste, refuse, rubbish, litter, scrap, flotsam and jetsam, rubble)
• IT Challenge – Lack of foundational data content/incorrect focus on technologies – Data is a resource – Growing awareness that it needs to be governed
• Knowledge Worker Challenge – The epitomy of knowledge work – Data is an asset – Most powerful, yet most underutilized and poorly managed asset
• What next? – A challenge to your organization
Data
• Overly dependent upon:
– Human-beings
– Wetwear
– Tribal knowledge
– Informal communications
– Non-repeatable practices
Data / Information Gap
Information
Copyright 2016 by Data Blueprint Slide # 35
Enron• August 2001 Enron stock falls to $42/share from $90/share
• Dynergy brings several $ billion in an attempted rescue
• Enron spends entire amount in 1 week
– Any person can write a check at Enron for
– Any amount of money for
– Any purchase at
– Any time
• Enron goes back to Dynergy for more $
• Dynergy: What happened to the several $ billion I gave you last week?
• Enron: – http://en.wikipedia.org/wiki/Enron
Copyright 2016 by Data Blueprint Slide # 36
What do we teach knowledge workers about data?
Copyright 2016 by Data Blueprint Slide # 37
What percentage of the deal with it daily?
What do we teach IT professionals about data?
Copyright 2016 by Data Blueprint Slide # 38
• 1 course
– How to build a new database
• What impressions do IT professionals get from this education?
– Data is a technical skill that is needed when developing new databases
• If we are migrating databases, we are not creating new databases and we don't need organizational data management knowledge, skills, and abilities (KSAs).
• If we are implementing a new software package, we are not creating a new database and therefore we do not need data management KSAs.
• If we are installing an enterprise resource package (ERP), we are not creating a new database and therefore we do not need data management KSAs.
Architecture and
Engineering
Architecture
Engineering
• Architecture enables complex "things" to be built
• Engineering ensures a disciplined approach to development
Copyright 2016 by Data Blueprint Slide # 39
Niccolo Machiavelli (1469-1527)
He who has not first laid his foundations may be able with great ability to lay them afterwards ... ... but they will be laid with trouble to the architect and danger to the builder.
Copyright 2016 by Data Blueprint Slide #
Machiavelli, Niccolo. The Prince. 19 Mar. 2004 http://pd.sparknotes.com/philosophy/prince
40
Copyright 2016 by Data Blueprint Slide # 41
You cannot architect after implementation!
Good Architectural Foundation?
Copyright 2016 by Data Blueprint Slide # 42
Poor Architectural Foundation
Copyright 2016 by Data Blueprint Slide # 43
USS Midway & Pancakes
• It is tall
• It has a clutch
• It was built in 1942
• It is still in regular use!
What is this?
Copyright 2016 by Data Blueprint Slide # 44
DAMA International Data Management Body of Knowledge
Copyright 2016 by Data Blueprint Slide # 45
(DM BoK)
Copyright 2016 by Data Blueprint Slide # 46
Data Management Maturity Model (DMM)
Copyright 2016 by Data Blueprint Slide # 47
Develop/Implement Software
Develop/Implement Data
This approach can only work when no sharing of
data occurs!
"Waterfall" and other SDLC models create data silos
Evolving Data is Different than Creating New SystemsCommon Organizational Data
(and corresponding data needs requirements)
New Organizational Capabilities
Systems Development
Activities
Create
Evolve
Future State
(Version +1)
Data evolution is separate from, external to, and precedes system development life cycle activities!
Copyright 2016 by Data Blueprint Slide # 48
My barn had to pass a foundation inspection …
• … before further construction could proceed • No IT equivalent
Copyright 2016 by Data Blueprint Slide # 49
Copyright 2016 by Data Blueprint Slide #
Managing Data wi th Guidance
What is Data Governance?
50
• Cash & other financial instruments • Real property • Inventory • Intellectual Property • Human
– Knowledge – Skills – Abilities
• Financial • Organizational reputation • Goodwill • Brand name • Data!!!
Many Examples of Organizational Assets
Copyright 2016 by Data Blueprint Slide # 51
Data Assets
Financial Assets
RealEstate Assets
Inventory Assets
Non-depletable
Available for subsequent use
Can be used up
Can be used up
Non-degrading √ √ Can degrade
over timeCan degrade
over time
Durable Non-taxed √ √
Strategic Asset √ √ √ √
We believe ...• Today, data is the most powerful, yet underutilized and poorly managed organizational asset • Data is your
– Sole – Non-depletable – Non-degrading – Durable – Strategic
• Asset – Data is the new oil! – Data is the new (s)oil! – Data is the new bacon!
• Our mission is to unlock business value by – Strengthening your data management capabilities – Providing tailored solutions, and – Building lasting partnerships
Copyright 2016 by Data Blueprint Slide # 52
Asset: A resource controlled by the organization as a result of past events or transactions and from which future economic benefits are expected to flow [Wikipedia]
Data Assets Win!
John Snow's 1854 Cholera Map of London
Copyright 2016 by Data Blueprint Slide # 53
While the basic elements of topography and theme existed previously in
cartography, the John Snow map was unique, using cartographic methods not
only to depict but also to analyze clusters of geographically dependent phenomena
• A certain university
• The Egg Man
• Admission Date
Three Examples
Copyright 2016 by Data Blueprint Slide # 54
What can Rolls Royce Learn• Old model
– Sell jet engines
• New model – Sell hours of thrust power – No payment for down time – Wing to wing
Copyright 2016 by Data Blueprint Slide # 55
from Nascar?
Anonymous Identifier Crime Sentence Gender Race Disability LEP …
1 92G-gqP-6ek-oy3 A 30 M W 0 N
2
3
First Name Last Name Date of Birth Zip Code Crime Sentence Gender Race Disability LEP …
1 Peter Aiken 1/17/1959 23192 A 30 M W 0 N
2
3
Hash Function(Replaces PII with Anonymous Identifier)
Hashing Process Illustrated
Copyright 2016 by Data Blueprint Slide # 56
Data Amalgamation Process
Copyright 2016 by Data Blueprint Slide # 57
Hash Function
CorrectionsDJJHealth DSS DMAS …
Hash Function
Hash Function
Hash Function
Hash Function
VCU Dataset (No PII)
Watson Analytics
Diagnosing Organizational Readiness
Copyright 2016 by Data Blueprint Slide # 58
adapted from the Managing Complex Change model by Dr. Mary Lippitt, 1987
Culture is the biggest impediment to a shift in
organizational thinking about data!
Copyright 2016 by Data Blueprint Slide # 59
PETER AIKEN WITH JUANITA BILLINGSFOREWORD BY JOHN BOTTEGA
MONETIZINGDATA MANAGEMENT
Unlocking the Value in Your Organization’sMost Important Asset.
10124 W. Broad Street, Suite C Glen Allen, Virginia 23060 804.521.4056
Copyright 2016 by Data Blueprint Slide #60