Peter Aiken and Michael Gorman The Case for the Chief Data ... · PDF...
Transcript of Peter Aiken and Michael Gorman The Case for the Chief Data ... · PDF...
Recasting the C-Suite to Leverage your most Valuable Asset
Presented by Peter Aiken, Ph.D.10124 W. Broad Street, Suite C
Glen Allen, Virginia 23060804.521.4056
The Case for the Chief Data Officer
Data
The Case for theChief Data O!cerRecasting the C-Suite to LeverageYour Most Valuable Asset
Peter Aiken andMichael Gorman
Copyright 2013 by Data Blueprint
2
Peter Aiken, PhD• 25+ years of experience in data
management• Multiple international awards &
recognition• Founder, Data Blueprint (datablueprint.com)
• Associate Professor of IS, VCU (vcu.edu)
• President, DAMA International (dama.org)
• 8 books and dozens of articles• Experienced w/ 500+ data
management practices in 20 countries• Multi-year immersions with
organizations as diverse as the US DoD, Nokia, Deutsche Bank, Wells Fargo, and the Commonwealth of Virginia
2
The Case for theChief Data O!cerRecasting the C-Suite to LeverageYour Most Valuable Asset
Peter Aiken andMichael Gorman
Copyright 2013 by Data Blueprint
CIOs
• Have accomplished astounding technological feats
• Have developed excellent organizational skill sets
• Have delivered phenomenal business value
3
Copyright 2013 by Data Blueprint
4
Copyright 2013 by Data Blueprint
5
Wordle.net
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
6
Copyright 2013 by Data Blueprint 7
Motivation• Organizations:
– $600 billion annually due to poor DM practices– Remain unaware of the root causes of their losses
• IT Professionals/Data Managers must:– Gain executive-level approval for basic DM investments– Must monetize these lost opportunities and related costs
• To avoid an unfortunate loop: – Management focused on fixing symptoms– Cannot address the underlying problems
• Business performance will be positively influenced• IT projects will better support business missions• Top management is more likely to support DM • Practitioners can prioritize relative IS investments
and adjust portfolios according to business needs• The bigger the organization, the more important
data assets are to the organization • This is due to the increased leverage obtainable
by larger organizations
Copyright 2013 by Data Blueprint
IT Project Failure Rates• Recent IT project failure rates statistics
can be summarized as follows: – Carr 1994
• 16% of IT Projects completed on time, within budget, with full functionality
– OASIG Study (1995)• 7 out of 10 IT projects "fail" in some respect
– The Chaos Report (1995)• 75% blew their schedules by 30% or more• 31% of projects will be canceled before they ever get completed• 53% of projects will cost over 189% of their original estimates• 16% for projects are completed on-time and on-budget
– KPMG Canada Survey (1997)• 61% of IT projects were deemed to have failed
– Conference Board Survey (2001) • Only 1 in 3 large IT project customers were very “satisfied"
– Robbins-Gioia Survey (2001)• 51% of respondents viewed their large IT implementation project as unsuccessful
– MacDonalds Innovate (2002)• Automate fast food network from fry temperature to # of burgers sold-$180M USD write-off
– Ford Everest (2004)• Replacing internal purchasing systems-$200 million over budget
– FBI (2005)• Blew $170M USD on suspected terrorist database-"start over from scratch" http://www.it-cortex.com/stat_failure_rate.htm (accessed 9/14/02)
New York Times 1/22/05 pA31
8
1 in 3 IT projects suffers on• Price• Schedule• Functionality
Copyright 2013 by Data Blueprint
IT Project Failure Rates (moving average)
9
Source: Standish Chaos Reports as reported at: http://www.galorath.com/wp/software-project-failure-costs-billions-better-estimation-planning-can-help.php
0%
15%
30%
45%
60%
1994 1993 1998 2000 2002 2004 2009
16%
27% 26%28%
34%
29%
32%
53%
33%
46%
49%51%
53%
44%
31%
40%
28%
23%
15%
18%
24%
Failed Challenged Succeeded
Copyright 2013 by Data Blueprint
"Most significant IT reform of the last decade"
10
1996 (passed)• Establish Agency CIOs
– Link IT investments to accomplishments
• Requires – CIO "Milestone Decision"
assessment– Establish process to select,
manage and control IT investments (CMM Level 2)
• Responsible– "developing, maintaining, and
facilitating the implementation of a sound and integrated information technology architecture"
Copyright 2013 by Data Blueprint
The Clinger-Cohen Act: 10 Years Later
11
2006 (assessed)• Mixed results
– Federal Enterprise Architecture
– "The landscape of federal information technology is not a clearly defined plain of reference points that can be empirically studied in pure isolation"
– Planned 5% decrease in IT costs– 9% increase instead
http://www.govexec.com/federal-news/2006/07/the-clinger-cohen-act-10-years-later/22227/
• Some guidance exists• Some experiences gained• Some progress has been made
£12bn NHS computer system is scrapped
Copyright 2013 by Data Blueprint
• The biggest civilian IT project of its kind in the world, it has already squandered at least £12.7billion. Some estimates put the cost far higher.
• Analysts say the sum would have paid the salaries of more than 60,000 nurses for a decade.
• Following an official review, the ‘one size fits all’ IT project will be replaced by much cheaper regional initiatives, with hospitals and GPs choosing the IT system they need.
• Read more: http://www.dailymail.co.uk/news/article-2040259/NHS-IT-project-failure-Labours-12bn-scheme-scrapped.html#ixzz2R1yb9F1i
12
Billion-Dollar Flop: Air Force Stumbles on Software Plan
Copyright 2013 by Data Blueprint
• In policy circles, problems that are mind-bogglingly difficult or impossible to solve, like global warming, are formally termed “wicked.”
• For the United States Air Force, installing a new software system has certainly proved to be a wicked problem. Last month, it canceled a six-year-old modernization effort that had eaten up more than $1 billion. When the Air Force realized that it would cost another $1 billion just to achieve one-quarter of the capabilities originally planned — and that even then the system would not be fully ready before 2020 — it decided to decamp.
13
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
14
Copyright 2013 by Data Blueprint
A Digital Universe ...• Consists of two species of bits:
– Differences in space– Differences in time– Logic can be done using zeros and
ones [Leibniz 1679]
• Structure bits– Vary in space but are invariant across time– These are memory (data)
• Sequence bits– Vary across time but are invariant across space– These are code
• While both are equally important to technology, more research and attention has been focused on the code than on the data [articulation from Dyson 2012]
15
Copyright 2013 by Data Blueprint
Observation
• Fred Brooks Jr.'s argumentthat data representation is the essence of programming. – "Show me your flowchart and conceal
your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious."
16
Copyright 2013 by Data Blueprint
17
Two Views of the Same Individual
Business are being bombardedoffers for these services
Copyright 2013 by Data Blueprint
Organizations Surveyed
18
• Results from more than 500 organizations
• 32% government• Appropriate
public company representation
• Enough data to demonstrate European organization DM practices are generally more mature
Local Government4%
State Government Agencies17%
Federal Government11%
Public Companies 58%
International Organizations10%
Copyright 2013 by Data Blueprint
Not Enough Data Management Involvement
19
Data Warehousing
XML
Data Quality
Customer Relationship Management
Master Data Management
Customer Data Integration
Enterprise Resource Planning
Enterprise Application Integration
Initiative Leader Initiative Involvement Not Involved
0
0.09
0.18
0.27
0.36
0.45
SuccessfulPartial Success
Don't know/too soon to tellUnsuccessful
Does not exist
• In 25 years:– "Successful" DM organizations fell from 43% to 15%– "Unsuccessful" increased from 5% to 21%.
Copyright 2013 by Data Blueprint
% of DM organizations labeled "successful"
20
19812007
Copyright 2013 by Data Blueprint
CIOs are distancing themselves from DM
21
1981 2007
Percentage Reporting Directly
to the CIO74% 43%
Percentage reporting 3 or
more levels below the CIO
26% 57%
Copyright 2013 by Data Blueprint
Why Data Projects Fail by Joseph R. Hudicka
• Assessed 1200 migration projects!
– Surveyed only experienced migration specialists who have done at least four migration projects
• The median project costs over 10 times the amount planned!
• Biggest Challenges: Bad Data; Missing Data; Duplicate Data
• The survey did not consider projects that were cancelled largely due to data migration difficulties
• "… problems are encountered rather than discovered"
$0 $125,000 $250,000 $375,000
Median Project Expense
Median Project Cost
Joseph R. Hudicka "Why ETL and Data Migration Projects Fail" Oracle Developers Technical Users Group Journal June 2005 pp. 29-31
22
Copyright 2013 by Data Blueprint
2/3rds have not formally established IA program
23
Forrester information architecture research
Copyright 2013 by Data Blueprint
24
4% have almost all they need from IA
Copyright 2013 by Data Blueprint
Root Cause Analysis
25
• Symptom of the problem– The weed– Above the surface – Obvious• The underlying
Cause– The root– Below the surface – Not obvious• Poor Information
Management Practices
Asking "why" repeatedly!
Copyright 2013 by Data Blueprint
Ishikawa Fishbone Diagrams
26
• Why is infant mortality so high?– Malnourished mothers• Why are mothers malnourished?
– Substandard biology educations in high school• Why do are biology programs substandard?
– Poor education of high school biology teachers• Why do we have poor biology teacher education?
– Biology profession unaware of consequences
• Why are so many organizational technology experiences so poor?–Misunderstanding of data's role in IT• Why do so few understand data's role in IT?
–Little, if any, focus on enterprise-wide data use in the educational system
• Why is the educational system not addressing this gap?–Lack of recognition by the system• Why has the system not yet been made aware
of this deficiency?–Lack of understanding at the C-level of these
issues• Why do they not understand?
–Little, if any, focus on enterprise-wide data use in the educational system
Copyright 2013 by Data Blueprint
Data InflationUnit Size What+it+means
Bit+(b) 1+or+0Short+for+“binary+digit”,+a=er+the+binary+code+(1+or+0)+computers+use+to+store+and+process+data
Byte+(B) 8+bitsEnough+informaCon+to+create+an+English+leEer+or+number+in+computer+code.+It+is+the+basic+unit+of+compuCng
Kilobyte+(KB) 1,000,+or+210,+bytes From+“thousand”+in+Greek.+One+page+of+typed+text+is+2KB
Megabyte+(MB) 1,000KB;+220+bytesFrom+“large”+in+Greek.+The+complete+works+of+Shakespeare+total+5MB.+A+typical+pop+song+is+about+4MB
Gigabyte+(GB) 1,000MB;+230+bytes From+“giant”+in+Greek.+A+twoWhour+film+can+be+compressed+into+1W2GB
Terabyte+(TB) 1,000GB;+240+bytesFrom+“monster”+in+Greek.+All+the+catalogued+books+in+America’s+Library+of+Congress+total+15TB
Petabyte+(PB) 1,000TB;+250+bytesAll+leEers+delivered+by+America’s+postal+service+this+year+will+amount+to+around+5PB.+Google+processes+around+1PB+every+hour
Exabyte+(EB) 1,000PB;+260+bytes Equivalent+to+10+billion+copies+of+The+Economist
ZeEabyte+(ZB) 1,000EB;+270+bytes The+total+amount+of+informaCon+in+existence+this+year+is+forecast+to+be+around+1.2ZB
YoEabyte+(YB) 1,000ZB;+280+bytes Currently+too+big+to+imagine
The prefixes are set by an intergovernmental group, the International Bureau of Weights and Measures. Source: The Economist Yotta and Zetta were added in 1991; terms for larger amounts have yet to be established
27
Driver:
Copyright 2013 by Data Blueprint
28
• Increasing demand• Increasing amount• Heightened public
and corporate sensitivity to security, privacy, and compliance
• Data dependent IT initiatives (BI, SOA, Analytics, Big Data ...)
• New data emphasis • Leveraging data for
competitive advantage and profitability
Copyright 2013 by Data Blueprint
29
A likely state of your data
Very
Silo
’ed or
conf
lictin
g data
sour
ces
Multiple Data Sources
Inconsistent data definitions of
common terms
IT are data owners
Lots of Data….Minim
um Inform
ation
Inconsistent Data Quality
Difficult to report and mine against
Redundancy
Multiple changes to source system
Copyright 2013 by Data Blueprint
30
A likely state of your data management efforts
1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
Copyright 2013 by Data Blueprint
5% Sales Increase Versus Data Volume
31
Sales Data Volume
Copyright 2013 by Data Blueprint
The Situation
32
Copyright 2013 by Data Blueprint
Aspirational Data in the Cloud
33
• This is a the approximate level of detail but let's next examine two extreme implementation examples 1) forklift 2) optimal 3) problems with above a) no basis for decisions made b) no inclusion of architecture/engineering concepts c) no idea that these concepts are missing from the process
Copyright 2013 by Data Blueprint
Getting into the Cloud
Transform
LessCleanerMore shareable ... data
34
• "We want to move our data management program to the next level"– Question: What level are you now?
• You are currently managing your data,– But, if you can't measure it, – How can you manage it effectively?
• How do you know where to put time, money, and energy so that data management best supports the mission?
Copyright 2013 by Data Blueprint
Strategic Motivation"One day Alice came to a fork in the road and saw a Cheshire cat in a tree. Which road do I take? she asked. Where do you want to go? was his response. I don't know, Alice answered. Then, said the cat, it doesn't matter."
Lewis Carroll from Alice in Wonderland
35
Copyright 2013 by Data Blueprint
36
Cruiser Collector
Copyright 2013 by Data Blueprint
Cruiser Collector
37
Data Program Coordination
Feedback
DataDevelopment
Copyright 2013 by Data Blueprint
StandardData
Organizational DM Practices Value-added WorkproductsOrganizational Strategies
Goals
BusinessData
Business Value
Application Models & Designs
Implementation
Direction
Guidance
38
OrganizationalData Integration
DataStewardship
Data SupportOperations
Data Asset Use
IntegratedModels
Leverage data in organizational activities
Data management processes andinfrastructure
Combining multipleassets to produceextra value
Organizational-entity subject area data
integration
Provide reliable data access
Achieve sharing of data within a business area
Copyright 2013 by Data Blueprint
Five Integrated DM Practice Areas
39
Manage data coherently.
Share data across boundaries.
Assign responsibilities for data.Engineer data delivery systems.
Maintain data availability.
Data Program Coordination
Organizational Data Integration
Data Stewardship Data Development
Data Support Operations
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0.4%
0.4%
0.3%0.3%
0%
0.5%
0%
1.0%
1.5%
5.4%
1.3%3.0%
4.9%4.9%
1.7%
9.8%8.6%
7.6%
15.7%
20.5%
Tota
l % o
f non
-rela
tiona
l pro
cess
ing
Percentage of non-relational processing (excluding mission-critical)Percentage of mission-critical, non-relational processing
Copyright 2013 by Data Blueprint
"The rumors of the demise of non-relational processing are greatly exaggerated" (Mark Twain)
40
• 68% using hierarchical (typically IMS or Adabase)
• 20% reporting operational network DBMS• Virtually no textbook education
Percentage of organizations relying on x amount of non-relational database processing
Copyright 2013 by Data Blueprint
5 Basic Data Structures
Indexed Sequential File: Built-in index permits location of records of persons with last names starting with "T"
Index
Program: Where is the record for person "Townsend?"
Index: Start looking here where the "Ts" are stored
Relational Database: Records are related to each other using relationships describable using relational algebra
Flat File: Records are typically sorted according to some criteria and must be searched from the beginning for each access
Program: Must start at the beginning and read each record when looking for
person "Townsend?"
Network Database: Records are related to each other using arranged master records associated with multiple detail records using linked lists and pointers
41
AssociativeConcept-orientedMulti-dimensional
Star schemaXML database
Hierarchical Database: Records are related to each other hierarchically using 'parent child' relationships
Data Program Coordination
Feedback
DataDevelopment
Copyright 2013 by Data Blueprint
StandardData
Organizational DM Practices and their Inter-relationshipsOrganizational Strategies
Goals
BusinessData
Business Value
Application Models & Designs
Implementation
Direction
GuidanceIdentifying, modeling, coordinating, organizing, distributing, and architecting data shared across business areas or organizational boundaries.
Ensuring that specific individuals are assigned the responsibility for the maintenance of specific data as organizational assets, and that those individuals are provided the requisite knowledge, skills, and abilities to accomplish these goals in conjunction with other data stewards in the organization.
Initiation, operation, tuning, maintenance, backup/recovery, archiving and disposal of data assets in support of organizational activities.
42
Specifying and designing appropriately architected data assets that are engineered to be capable of supporting organizational needs.
OrganizationalData Integration
DataStewardship
Data SupportOperations
Data Asset Use
IntegratedModels
Defining, coordinating, resourcing, implementing, and monitoring organizational data program strategies, policies, plans, etc. as coherent set of activities.
Data Program Coordination
Feedback
DataDevelopment
Copyright 2013 by Data Blueprint
StandardData
Organizational DM Practices Value-added WorkproductsOrganizational Strategies
Goals
BusinessData
Business Value
Application Models & Designs
Implementation
Direction
Guidance
43
OrganizationalData Integration
DataStewardship
Data SupportOperations
Data Asset Use
IntegratedModels
Leverage data in organizational activities
Data management processes andinfrastructure
Combining multipleassets to produceextra value
Organizational-entity subject area data
integration
Provide reliable data access
Achieve sharing of data within a business area
Data becomes a fuel!
Copyright 2013 by Data Blueprint
Various Maturity Frameworks
44
Usage basis Make it Happen
Make it Happen Faster
What Happened
?
Why did it happen?
What will happen?
Make it happen by
itself
What do I want to
happen?
How do we make it
happen better?
What should we do next?
Content basis Events Trans- actions Reporting Analyzing Predictive Operation-
alizeClosed
loopCollabor -
ative Foresight
Capability basis Initial Defined
Organization Basis Innovate Operate: Consolidate OptimizeIntegrate
OptimizedManagedRepeatable
Data is a lubricant!
Adapted from John Ladley
Copyright 2013 by Data Blueprint
Data Management Capability Maturity Model Levels
Our DM practices are ad hoc and dependent upon "heroes" and heroic efforts
Initial(1)
Repeatable(2)
We have DM experience and have the ability to
implement disciplined processes
We have experience that we have standardized so that all in
the organization can follow itDefined
(3)
Managed(4)
We manage our DM processes so that the whole organization can follow our standard
DM guidance
Optimizing(5)We have a process for improving our DM capabilities
One concept for process improvement, others include:
• Norton Stage Theory• TQM• TQdM• TDQM• ISO 9000
and focus on understanding current processes and determining where to make improvements.
Copyright 2013 by Data Blueprint
Assessment Components
Data Management Practice AreasData Management Practice Areas
Data program coordination
DM is practiced as a coherent and coordinated set of activities
Organizational data integration
Delivery of data is support of organizational objectives – the currency of DM
Data stewardshipDesignating specific individuals caretakers for certain data
Data development Efficient delivery of data via appropriate channels
Data support Ensuring reliable access to data
Capability Maturity Model Levels Examples of practice maturity
1 – InitialOur DM practices are ad hoc and dependent upon "heroes" and heroic efforts
2 - RepeatableWe have DM experience and have the ability to implement disciplined processes
3 - DocumentedWe have standardized DM practices so that all in the organization can perform it with uniform quality
4 - ManagedWe manage our DM processes so that the whole organization can follow our standard DM guidance
5 - Optimizing We have a process for improving our DM capabilities
46
Copyright 2013 by Data Blueprint
• CMU's Software Engineering Institute (SEI) Collaboration
• Results from hundreds organizations in various industries including:✓ Public Companies ✓ State Government Agencies✓ Federal Government✓ International Organizations
• Defined industry standard• Steps toward defining data management
"state of the practice"
Data Management Practices Measurement (DMPA)
47
Data Program Coordination
Organizational Data Integration
Data Stewardship
Data Development
Data Support Operations
Focus: Implementation
and Access
Focus: Guidance and
Facilitation
Optimizing (V)
Managed (IV)
Documented (III)
Repeatable (II)
Initial (I)
Client Industry Competition All Respondents
Copyright 2013 by Data Blueprint
Development guidance
Data Adminstration
Support systems
Asset recovery capability
Development training
0 1 2 3 4 5
Data Management Practices Assessment
Challenge
Challenge
Challenge
Result 1
Result 2
Result 3
Result 4
Result 5
48
Copyright 2013 by Data Blueprint
49
Copyright 2013 by Data Blueprint
Comparison of DM Maturity 2007-2012
50
1
2
3
4
5
Data
Prog
ram
Coor
dinati
on
Orga
nizati
onal
Data
Integ
ratio
n
Data
Stew
ards
hip
Data
Deve
lopme
nt
Data
Supp
ort O
pera
tions
2007 Maturity Levels 2012 Maturity Levels
Copyright 2013 by Data Blueprint
Conclusion must be?1. CIOs are unaware of the
strategic nature of data; or
2. CIOs are not concerned about how data management is accomplished in their organizations; or
3. CIOs think data management is being adequately accomplished in their organizations
51
• Approximately, 10% percent of organizations achieve parity and (potential positive returns) on their DM investments
• Only 30% of DM investments achieve tangible returns at all
• Seventy percent of organizations have very small or no tangible return on their DM investments
Copyright 2013 by Data Blueprint
Largely Ineffective Investments
52
Investment <= Return10%
Investment > Return20%
Return ≈ 070%
Copyright 2013 by Data Blueprint
53
• 60 GB of data/second• 200,000 hours of big data
will be generated testing systems
• 2,000 hours media coverage/daily
• 845 million Facebook users averaging 15 TB/day
• 13,000 tweets/second• 4 billion watching• 8.5 billion devices
connected
2012 London Summer Games
• Faster processors outstripped not only the hard disk, but main memory – Hard disk too slow– Memory too small
• Flash drives remove both bottlenecks– Combined Apple and Yahoo have
spend more than $500 million to date
• Make it look like traditional storage or more system memory– Minimum 10x improvements– Dragonstone server is 3.2 tb flash
memory (Facebook)
• Bottom line - new capabilities!
Copyright 2013 by Data Blueprint
"There’s now a blurring between the storage world and the memory world"
54
• von Neumann bottleneck (computer science)– "An inefficiency inherent in
the design of any von Neumann machine that arises from the fact that most computer time is spent in moving information between storage and the central processing unit rather than operating on it"[http://encyclopedia2.thefreedictionary.com/von+Neumann+bottleneck]
• Michael Stonebraker– Ingres (Berkeley/MIT)– Modern database
processing is approximately 4% efficient
• Many "big data architectures are attempts to address this, but:– Zero sum game– Trade characteristics
against each other• Reliability• Predictability
– Google/MapReduce/Bigtable
– Amazon/Dynamo– Netflix/Chaos Monkey– Hadoop– McDipper
• Big data exploits non-von Neumann processing
Copyright 2013 by Data Blueprint
Non-von Neumann Processing/Efficiencies
55
Copyright 2013 by Data Blueprint
56
IBM's Data Baby
Copyright 2013 by Data Blueprint
57
A likely state of your data
Very
Silo
’ed or
conf
lictin
g data
sour
ces
Multiple Data Sources
Inconsistent data definitions of
common terms
IT are data owners
Lots of Data….Minim
um Inform
ation
Inconsistent Data Quality
Difficult to report and mine against
Redundancy
Multiple changes to source system
• 5 Data management practices areas / data management basics ...
• ... are necessary but insufficient prerequisites to organizational data leveraging applications that is self actualizing data or advanced data practices
Copyright 2013 by Data Blueprint
Advanced Data Practices• Cloud• MDM• Mining• Big Data• Analytics• Warehousing• SOA
Hierarchy of Data Management Practices (after Maslow)
Basic Data Management Practices– Data Program Management– Organizational Data Integration– Data Stewardship– Data Development– Data Support Operations
http://3.bp.blogspot.com/-ptl-9mAieuQ/T-idBt1YFmI/AAAAAAAABgw/Ib-nVkMmMEQ/s1600/maslows_hierarchy_of_needs.png
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
59
Copyright 2013 by Data Blueprint
Application-Centric Development
Original articulation from Doug Bagley @ Walmart
60
t
t
Strategy
Goals/Objectives
Systems/Applications
Network/Infrastructure
Data/Information
t
• In support of strategy, the organization develops specific goals/objectives
• The goals/objectives drive the development of specific systems/applications
• Development of systems/applications leads to network/infrastructure requirements
• Data/information are typically considered after the systems/applications and network/infrastructure have been articulated
• Problems with this approach:– Ensures that data is formed
around the application and not the information requirements
– Process are narrowly formed around applications
– Very little data reuse is possible
Copyright 2013 by Data Blueprint
Payroll Application(3rd GL)Payroll Data
(database)
R& D Applications(researcher supported, no documentation)
R & DData(raw) Mfg. Data
(home growndatabase)
Mfg. Applications(contractor supported)
FinanceData
(indexed)
Finance Application(3rd GL, batch
system, no source)
Marketing Application(4rd GL, query facilities, no reporting, very large)
Marketing Data(external database)
Personnel App.(20 years old,
un-normalized data)
Personnel Data(database)
61
Typical System Evolution
Copyright 2013 by Data Blueprint
Silos
• Not an acronym for SIngle LOcation
62
Copyright 2013 by Data Blueprint
History Lesson
• "In the first decades of computing, the programs in a corporation became an unruly mess, far removed from the orderliness one would normally associate with an engineering discipline." – James Martin 1981
63
Copyright 2013 by Data Blueprint
Comments from CIOs• What a Mess!• CIOs have demanding jobs as information
systems in an organization are often taken for granted until something breaks down.
• The CIO is responsible for explaining to executive management the complex nightmare this industry has gotten itself into over the past 40 years and why equipment must be constantly retrofitted or replaced.
64
Einstein Quote
Copyright 2013 by Data Blueprint
65
"The significant problems we face cannot be solved at the same level of thinking we were at when we created them."- Albert Einstein
Copyright 2013 by Data Blueprint
What does it mean to treat data as an organizational asset?• Assets are economic resources
– Must own or control– Must use to produce value– Value can be converted into cash
• An asset is a resource controlled by the organization as a result of past events or transactions and from which future economic benefits are expected to flow to the organization [Wikipedia]
• With assets:– Formalize the care and feeding of data
• Cash management - HR planning
– Put data to work in unique and significant ways• Identify data the organization will need
[Redman 2008]
66
Copyright 2013 by Data Blueprint
Data-Centric Development Flow
Original articulation from Doug Bagley @ Walmart
67
t
t
Strategy
Goals/Objectives
Data/Information
Network/Infrastructure
Systems/Applications
t
• In support of strategy, the organization develops specific goals/objectives
• The goals/objectives drive the development of specific data/information assets with an eye to organization-wide usage
• Network/infrastructure components are developed to support organization-wide use of data
• Development of systems/applications is derived from the data/network architecture
• Advantages of this approach:– Data/information assets are
developed from an organization-wide perspective
– Systems support organizational data needs and compliment organizational process flows
– Maximum data/information reuse
Copyright 2013 by Data Blueprint
68
Copyright 2013 by Data Blueprint
Data Centric Principles1. Help organizations prepare for future change by implementing a flexible and
adaptable organizational data architecture;2. Focus data assets to efficiently and effectively support organizational strategy;3. Increase the percentage of time available to accomplish this by a lower
percentage of time on maintenance activities;4. Reducing organizational data ROT;5. Remaining data will receive more "attention" with respect to quality/security/reuse;6. Reducing the amount and complexity of the organizational code-base;7. Reducing the amount of time and efforts and risk associated with IT projects;8. Engineer flexibility and adaptability into data architectures instead of attempting to
retrofit them in after they are in production;9. Produce more, reusable data-focused work products;10.When faced with a choice between chaos versus understanding, organizations will
gravitate towards a cheaper, more understandable solution;11.Same comment when comparing complexity versus ease of implementation;12.Decrease the time spend understanding versus time spent considering the data-
focused portions of organizational strategy;13.Uncertain benefits versus engineerable benefits.
69
Copyright 2013 by Data Blueprint
Data Centric Principles• Faster
– Planning for data-focused aspects of IT will require less fact-finding/guesswork
– Many data transformations can be "ironed out" of processing
– Error correction is faster due to less data volume/more understandable data
• Better– Quality of the remaining data can be more easily improved– Resulting data architecture will be more understandable/responsive– Strategy can be explicitly linked to KPIs (or whatever)
• Cheaper– Maintain only what is important – can experience 80% reduction in volume– Less data leads to fewer physical processing/governance requirements– Less diagnostic/planning time required to address challenges
70
Strategy
Goals/Objectives
Data/Information
Network/Infrastructure
Systems/Applications
Original articulation from Doug Bagley @ Walmart
Strategy
Goals/Objectives
Systems/Applications
Network/Infrastructure
Data/Information
Copyright 2013 by Data Blueprint
This represents a gradual shift from
application to data-centric
71
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
72
Copyright 2013 by Data Blueprint
History (such as it is)• Automate existing manual processing • Data management was simple
– Running millions of punched cards through banks of sorting, collating and tabulating machines
– Results printed on paper or punched onto more cards
– Data management meant physically storing and hauling around punched cards
• Tasks (check signing, calculating, and machine control) were implemented to provide automated support for departmental-based processing
• Creating information silos
73
Copyright 2013 by Data Blueprint
74
Copyright 2013 by Data Blueprint
Data Processing Manager
75
• Fifty years ago, data management was simple. Data processing meant running millions of punched cards through banks of sorting, collating and tabulating machines, with the results being printed on paper or punched onto still more cards. And data management meant physically storing and hauling around all those punched cards (Hayes 2002).
Copyright 2013 by Data Blueprint
Chief Information Officer
76
CIO Responsibilities
Copyright 2013 by Data Blueprint
77
• A job title commonly given to the most senior executive in an enterprise responsible for the information technology and computer systems that support enterprise goals
Copyright 2013 by Data Blueprint
CFO Necessary Prerequisites/Qualifications• CPA
• CMA
• Masters of Accountancy
• Other recognized degrees/certifications
• These are necessary but insufficient prerequisites/qualifications
78
Copyright 2013 by Data Blueprint
CIO Qualifications• No specific qualifications• Typically technological fields: computer science,
software engineering, or information systems • Many have Master of Business Administration or
Master of Science in Management degrees• Recently CIOs' leadership capabilities, business
acumen and strategic perspectives have taken precedence over technical skills.
• It is now quite common for CIOs to be appointed from the business side of the organization, especially if they have project management skills.
79
Copyright 2013 by Data Blueprint
Qualifications
Leadership capabilities, business acumen and strategic perspectives have taken precedence over technical skills
80
Data
Copyright 2013 by Data Blueprint
What do we teach knowledge workers about data?
81
What percentage of the deal with it daily?
Copyright 2013 by Data Blueprint
What do we teach IT professionals about data?• 1 course
– How to build a new database
– 80% if UT expenses are used to improve existing IT assets
• What impressions do IT professionals get from this education?– Data is a technical
skill that is used to develop new databases
• This is not the best way to educate IT and business professionals - every organization's– Sole, non-depletable,
non-degrading, durable, strategic asset
82
Associate Productivity
Customer Insights
Human Capital Corp. Reputation Acquisition Strategic Planning
Real estate CRM CRM
Analytic and reporting processes
Corporate Reputation - Risk Management, Compliance, Marketing, IT and Data Governance
Corporate Processes
Corporate Data
Inventory Mgmt
Tran
sfor
mat
ion
Por
tfol
io
Supply Chain
Multi ChannelMerchant Tools Supply Chain
Strategic Initiatives
AcctingSales
Transactional Processing
Logistics Associate Locations and Codes
Item
Customer Suppliers
Retail Planning
Copyright 2013 by Data Blueprint
No clear connection exists between to business priorities and IT initiatives
83
Grow expenses slower than
sales
Grow operating income faster
than sales
Pass on savings
Drive efficiency with technology
Leverage scale globally
Leverage expertise
Deploy new formats
Grow productivity of existing assets
Attract new members
Expand into new channels
Enter new markets
Make acquisitions
Produce significant free
cash flow
Drive ROI performance
Deliver greater shareholder
value C
usto
mer
P
ersp
ectiv
e Open new stores
Develop new, innovative formats
Appeal to new demographics
Integrate shopping
experience
Develop new, innovative formats
Remain relevant to all
customers
Increase "Green" Image
Inte
rnal
P
ersp
ectiv
e
Create competitive advantages
Improve use of information
Strengthen supply chain
Improve Associate
productivity
Making acquisitions
Increase benefit from our global expertise
Present consistent view and
experience
Integrate channels Match staffing
to store needs Increase sell through
Fina
ncia
l P
ersp
ectiv
e Reduce expenses
Inventory Management
Human and Intell. Capital investment
Manage new facilities
Improve Sales and margin by facilities
Increased member-base
revenues
Revenue growth Cash flow Return on
Capital
Walmart Strategy Map
See more uniform brand and retail experience
Leverage Growth Return
Gross Margin Improvement
CE
O P
ersp
ectiv
e
Attract more customers & have customer purchasing more
( Alignment Gap )
Adapted from John Ladley
Data
Copyright 2013 by Data Blueprint
Comments from CIOs• On CIOs as strategic business partners
• A small percentage of CIO’s have truly attained that status, despite many who claim they have. I would guess fewer than 10% of the CIO population achieves that goal. Perhaps we are expecting the CIO to wear too many hats. When they fail to effectively perform in all of those roles their time in position is limited, which might help explain why CIO longevity is so short (CIO = “Career Is Over”).
84
Copyright 2013 by Data Blueprint
Q1Keeping the doors open
(little or no proactive data management)
Q2Increasing organizational efficiencies/effectiveness
Q3Using data to create
strategic opportunitiesQ4
Both(Cash Cow)
Improve Operations
Inno
vatio
n
Only 1 is 10 organizations has a board approved data strategy!
Enterprise Data Strategy Choices
Copyright 2013 by Data Blueprint
Innovation• Innovation is the development of new customers value
through solutions that meet new needs, inarticulate needs, or old customer and market needs in new ways. This is accomplished through different or more effective products, processes, services, technologies, or ideas that are readily available to markets, governments, and society.
• Innovation differs from invention in that innovation refers to the use of a better and, as a result, novel idea or method, whereas invention refers more directly to the creation of the idea or method itself.
• Innovation differs from improvement in that innovation refers to the notion of doing something different (Lat. innovare: "to change") rather than doing the same thing better.
86
Copyright 2013 by Data Blueprint
87
Bills of Mortality by Captain John Graunt
Copyright 2013 by Data Blueprint
88
Bills of Mortality
Copyright 2013 by Data Blueprint
89Mortality Geocoding
Where is it happening?
Copyright 2013 by Data Blueprint
90
Plague Peak
When is it happening?
("Whereas of the Plague")
Copyright 2013 by Data Blueprint
91
Black Rats or Rattus Rattus
Why is it happening?
Black Rats or Rattus Rattus
Why is it happening?
Copyright 2013 by Data Blueprint
92
What will happen?
Copyright 2013 by Data Blueprint
93
John Snow's 1854 Cholera Map of London
Copyright 2013 by Data Blueprint
International Chemical Company Engine Testing
94
• $1billion (+) chemical company
• Develops/manufactures additives enhancing the performance of oils and fuels ...
• ... to enhance engine/machine performance – Helps fuels burn cleaner– Engines run smoother– Machines last longer
• Tens of thousands of tests annually– Test costs range up to
$250,000!
95
1.Manual transfer of digital data2.Manual file movement/duplication3.Manual data manipulation4.Disparate synonym reconciliation 5.Tribal knowledge requirements 6.Non-sustainable technology
Copyright 2013 by Data Blueprint
Overview of Existing Data Management Process
Copyright 2013 by Data Blueprint
Data Integration Solution• Integrated the existing systems to
easily search on and find similar or identical tests
• Results:– Reduced expenses– Improved competitive edge
and customer service– Time savings and improve operational
capabilities
• According to our client’s internal business case development, they expect to realize a $25 million gain each year thanks to this data integration
96
Copyright 2013 by Data Blueprint
Data Data
Data
Information
Fact Meaning
Request
A Model Specifying Relationships Among Important Terms
[Built on definition by Dan Appleton 1983]
Intelligence
Use
1. Each FACT combines with one or more MEANINGS. 2. Each specific FACT and MEANING combination is referred to as a DATUM. 3. An INFORMATION is one or more DATA that are returned in response to a specific REQUEST 4. INFORMATION REUSE is enabled when one FACT is combined with more than one
MEANING.5. INTELLIGENCE is INFORMATION associated with its USES.
Wisdom & knowledge are often used synonymously
Data
Data
Data Data
97
Copyright 2013 by Data Blueprint
Data Leverage
• Permits organizations to better manage their sole non-depleteable, non-degrading, durable, strategic asset - data– within the organization, and – with organizational data exchange partners
• Leverage – Obtained by implementation of data-centric technologies, processes, and human skill
sets– Increased by elimination of data ROT (redundant, obsolete, or trivial)
• The bigger the organization, the greater potential leverage exists
• Treating data more asset-like simultaneously 1. lowers organizational IT costs and 2. increases organizational knowledge worker productivity
98
Less ROT
Technologies
Process
People
He who doesn’t lay his foundations before hand, may by great abilities do so afterward ...... although with great trouble to the architect and danger to the building.
Copyright 2013 by Data Blueprint
Machiavelli, Niccolo. The Prince. 19 Mar. 2004 http://pd.sparknotes.com/philosophy/prince
99
Niccolo Machiavelli (1469-1527)
You cannot architect after implementation!
Copyright 2013 by Data Blueprint
100
USS Midway & Pancakes
Copyright 2013 by Data Blueprint
101
What is this?
• It is tall• It has a clutch• It was built in 1942• It is still in regular use!
Copyright 2013 by Data Blueprint
102
http://www.youtube.com/watch?v=Hdpf-MQM9vY&feature=player_embedded#!
360 hours or 15 days of continuous buildingThe Role of Engineering/Architecture in Rapid Development
Data Architectures are Developed in Response to Organizational Needs
! ! ! !
Copyright 2013 by Data Blueprint
103
Organizational Needs
become instantiated and integrated into an Data/Information
Architecture
Informa(on)System)Requirements
authorizes and articulates sa
tisfy
spe
cific
org
aniz
atio
nal n
eeds
Copyright 2013 by Data Blueprint
An organization's data architecture ...
104
Software Package 1
Software Package 2
Software Package 3
Software Package 4
Software Package 5
Software Package 6
Data Architecture
... maps between and across software packages
Copyright 2013 by Data Blueprint
Comments from CIOs• Most CIO’s today are challenged with
being “experts” on technology (infrastructure and application), business process, relationship management and data management). None are successful at all and most have a bent towards only one of those areas, depending upon where they began their career and the path they took to attain their CIO role.
105
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
106
Copyright 2013 by Data Blueprint
Out of degree and intensity, grows the organizational "data challenge"
1. Complexity
Trivial to complex
2. Intensity
Degree
107
Data Challenge
Complexity
Inte
nsity
Copyright 2013 by Data Blueprint
108
Copyright 2013 by Data Blueprint
CIO's are generally unsuccessful at remaining data-focused
• Question:– What is the hardest part of doing analysis?
• Answer:– Not doing design!
• Question:– What is the hardest part of a
CIO's job?• Answer:
– Remaining data focused!• Anything the CIO does that is not ...
– ... applying organizational data assets to the implementation of organizational strategy ...... is a similar distraction
109
Copyright 2013 by Data Blueprint
Someone Notable Thinks DM is Important
110
excerpt from President Obama's 2012 State of the Union Address
Copyright 2013 by Data Blueprint
58 Commonly Used Chief Officer Titles
111
• Chief Accounting Officer, Chief Administrative Officer, Chief Analytics Officer, Chief Audit Officer, Chief Brand Officer, Chief Business Officer, Chief Channel Officer, Chief Commercial Officer, Chief Communications Officer, Chief Compliance Officer, Chief Creative Officer, Chief Data Officer, Chief Executive Officer, Chief Financial Officer, Chief Human Resources Officer, Chief Information Officer, Chief Information Security Officer, Chief Innovation Officer, Chief Investment Officer, Chief Immigration Officer, Chief Geospatial Information Officer, Chief Knowledge Officer, Chief Leadership Officer, Chief Learning Officer, Chief Legal Officer, Chief Marketing Officer, Chief Marketing Information Officer, Chief Medical Officer, Chief Merchandising Officer, Chief Networking Officer, Chief Operating Officer, Chief Process Officer, Chief Procurement Officer, Chief Product Officer, Chief Research Information Officer, Chief Risk Officer, Chief Science Officer, Chief Stores Officer, Chief Strategy Officer, Chief Technology Officer, Chief Visionary Officer, Chief Web Officer
Copyright 2013 by Data Blueprint
The "Chief Officer" Title
112
• Chief– The head or leader of an organized body of people; the person
highest in authority: the chief of police
• Chief Financial Officer (CFO)– Individual possessing the knowledge, skills, and abilities to be both
the final authority and decision-maker in organizational financial matters
• Chief Risk Officer (CRO)– Individual possessing the knowledge, skills, and abilities makes
decisions and implements risk management
• Chief Medical Officer (CMO)– Responsible for organizational medical matters. The organization,
and the public, has similar expectations for any of chief officer – especially after the Sarbanes-Oxley bill.
[dictionary.com]
ChiefCopyright 2013 by Data Blueprint
113
C-levelCopyright 2013 by Data Blueprint
114
C OXCopyright 2013 by Data Blueprint
115
ESPOQR
Copyright 2013 by Data Blueprint
58 Commonly Used Chief Officer Titles
116
• Chief Accounting Officer, Chief Administrative Officer, Chief Analytics Officer, Chief Audit Officer, Chief Brand Officer, Chief Business Officer, Chief Channel Officer, Chief Commercial Officer, Chief Communications Officer, Chief Compliance Officer, Chief Creative Officer, Chief Data Officer, Chief Executive Officer, Chief Financial Officer, Chief Human Resources Officer, Chief Information Officer, Chief Information Security Officer, Chief Innovation Officer, Chief Investment Officer, Chief Immigration Officer, Chief Geospatial Information Officer, Chief Knowledge Officer, Chief Leadership Officer, Chief Learning Officer, Chief Legal Officer, Chief Marketing Officer, Chief Marketing Information Officer, Chief Medical Officer, Chief Merchandising Officer, Chief Networking Officer, Chief Operating Officer, Chief Process Officer, Chief Procurement Officer, Chief Product Officer, Chief Research Information Officer, Chief Risk Officer, Chief Science Officer, Chief Stores Officer, Chief Strategy Officer, Chief Technology Officer, Chief Visionary Officer, Chief Web Officer
Copyright 2013 by Data Blueprint
The "Chief Officer" Title
117
• Chief– The head or leader of an organized body of people; the person
highest in authority: the chief of police
• Chief Financial Officer (CFO)– Individual possessing the knowledge, skills, and abilities to be both
the final authority and decision-maker in organizational financial matters
• Chief Risk Officer (CRO)– Individual possessing the knowledge, skills, and abilities makes
decisions and implements risk management
• Chief Medical Officer (CMO)– Responsible for organizational medical matters. The organization,
and the public, has similar expectations for any of chief officer – especially after the Sarbanes-Oxley bill.
[dictionary.com]
Copyright 2013 by Data Blueprint
Skills Desired of CIOs• Technical/business
competencies • Timely and effective
execution• Being collaborative• Business knowledge• Creating a strategic vision• Inspire/leadership
responsibilities• Politically astute• Business acumen• Know of funding flows and
critical levers • Human capital management
• Talent evaluation, development, goal-setting and performance management
• Strategic-value creation• Creating revenue-generating
opportunities • Leadership• Influence others through
consensus building, storytelling, communications, modeling
• Networking, self-promotion, negotiating, empathy
118
Data
Copyright 2013 by Data Blueprint
CIO Concerns 2007
1. People leadership2. Managing budgets3. Business alignment4. Infrastructure refresh5. Security
6. Compliance7. Resource management8. Managing customers9. Managing change10. Board politics
119
Data
Copyright 2013 by Data Blueprint
Gartner 2008 CIO Business & Technology Priorities
120
Top 10 Business Priorities Top 10 Technology Priorities
Business process improvement Business intelligence applications
Attracting and retaining new customers Enterprise applications (ERP, CRM, etc.)
Creating innovative products and services Servers and storage technologies
Expanding into new markets or geographies Legacy modernization/enhancement
Reducing enterprise costs Technical infrastructure
Improving enterprise workforce effectiveness Security technologies
Expanding current customer relationships Networking, voice and data
Increasing the use of information/analytics Collaboration technologies
Targeting customers/markets more effectively Document management
Acquiring new companies/capabilities Service-oriented architecture SOA
Data
1. Integrating systems and processes2. Strategic planning/aligning IT and organizational goals3. Project management improvements4. Security and privacy measures5. Lowering costs6. Line of business initiatives7. Staff development/retention/recruitment
Copyright 2013 by Data Blueprint
Initiatives that will provide greatest value in FY 2008
121
TechAmerica’s Nineteenth Annual Survey of Federal Chief Information Officers February 2009
Data
83.00%
76.00%
71.00%
68.00%
68.00%
66.00%
64.00%
64.00%
61.00%
60.00%
BI/Analytics
Virtualization
Governance/Risk/Compliance
Customer Collaboration
Mobility Solutions
Self-service Portals
Applications Harmonization
Business Process Management
Service-oriented Architecture
Unified Communications
Copyright 2013 by Data Blueprint
CIOs have visionary plans/CIO innovation is not limited to IT solutions
122
Data
Copyright 2013 by Data Blueprint
2010 Gartner Top 10 CIO Technology Priorities1. Cloud computing2. Virtualization3. Mobile technologies4. IT management5. Business intelligence6. Networking7. Voice and data communications8. Enterprise applications9. Collaboration technologies10.Infrastructure, and Web 2.0
123
Data
Copyright 2013 by Data Blueprint
Global CIO: The Top 10 Issues For 20111. Seeing And Shaping The Future: Analytics2. The Budget Trap Becomes The Competitive-Performance Gap:
Doing more with less3. The iPad Explosion: Mobile Strategies4. Digitizing The Enterprise5. Social Media: From Grudging Acceptance To Hair-On-Fire
Evangelism6. Customer Engagement Soars To
Unprecedented Levels7. Enabling The Massively Adaptable
Data Center8. The CIO As Chief Acceleration Officer9. The Importance Of Being Global10.Optimizing Opportunities With Optimized Systems
http://www.informationweek.com/news/global-cio/interviews/229000361?pgno=1
124
Data
Copyright 2013 by Data Blueprint
CIO Roles
125
Data
Copyright 2013 by Data Blueprint
CIO's top 3 selection of business strategies in 2011 and projected for 2014
126
Data
Copyright 2013 by Data Blueprint
CIO Infrastructure Focus
Help DeskMobileDesktop
User N
eeds
Conne
ctivit
y
Networking Telephony
Back
End
Sys
.
VirtualizationCloud SupportData Centers
127
Copyright 2013 by Data Blueprint
Comments from CIOs• Gartner has been pontificating on the
evolution of the CIO role towards CPO – Chief Process Officer. So now the CIO would own all technology, all processes and all data. No other organization is experiencing this evolution into other spheres of influence. The CHRO does HR work. The CFO does financial work. The COO does operations work. However, the CIO is expected to be the head of technology, the architect of all business processes and the intelligence behind leveraging data.
128
Copyright 2013 by Data Blueprint
Top Five CIO Concerns 2005-2011
129
2005 2006 2007 2008 2009 2010 2011Grant Thornton-State CIO Survey 1 1 1 1 1 1 6Ameritech ITAA Annual CIO survey 1 1 1 1 1 1 6CIO Magazine-State of the CIO 1 1 1 1 1 5UK CIO Survey 1 1Gartner Annual Priorities 1 1 1 1 1 1 1 7Informationweek Global CIO top 10 issues 1 1 1 3Accenture CIO Survey 1 1KPMG & Harvey Nash 1 1 1 1 1 1 6NASCIO Survey 1 1 1 1 1 1 1 7Robert Half Technology 1 1 1 1 1 1 6
3 6 9 7 8 8 7 48
2005 2006 2007 2008 2009 2010 2011 0.000
0.200
0.400
0.600
0.800
IT/Infor
mation S
ecurity
/Privacy
Virtualiz
ation
Data ce
nter/IT
effici
encie
s/Clou
d
Social M
edia
Impro
ving p
eople
/leade
rship
BI/ana
lytics
Standa
rdizat
ion/co
nsolida
tion
IT workfor
ce de
velop
ment
IT gover
nance
Risk m
anag
emen
t
Mobile
applic
ations/
techn
ologie
s
Inform
ation S
harin
g
Imple
menting
plans/
initativ
es/ach
ieving
resul
ts
Acquisit
ion/pr
oject m
gt
Process
/syste
m integ
ration
Strateg
ic plan
ning
Copyright 2013 by Data Blueprint
Top Five CIO Concerns 2005-2011
130
Copyright 2013 by Data Blueprint
131
CIOs aren't ...• The ultimate authority
on organizational informational assets
• Able to devote the required time/attention to the management of organizational informational assets
• Possessed of the requisite expertise to manage organizational informational assets
• Situated to achieve success organizationally as long as they have a technology development perspective
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
132
projectcartoon.comCopyright 2013 by Data Blueprint
• Original business concept• As the consultant described it• As the customer explained it• How the project leader
understood it• How the programmer wrote it• What the beta testers received• What operations installed• As accredited for operation• When it was delivered• How the project was documented• How the help desk supported it• How the customer was billed• After patches were applied• What the customer wanted
Traditional Systems Life Cycle Challenges
133
SystemRequirements
SoftwareRequirements
PreliminaryDesign
DetailedDesign
Coding &De-Bugging
Integration& Testing
Operations &Maintenance
"Waterfall" model
134Copyright 2013 by Data Blueprint
46
Copyright 2013 by Data Blueprint
Correctimplementations
Correctfunctionality
Correct designs
Correct specifications
Implementations based on
erroneous design
Implementations based on
erroneous specs
Incorrect implementations
Uncorrectableerrors
Hiddenerrors
Correctable functionality
(Adapted from [Mizuno 1983] as reproduced by Davis 1990.)
Design
Implementation
Requirements
Testing
imperfect program products
Cumulative Effect of Errors in Systems
Development
Erroneous designs
Erroneous specifications
the "real" problem
Designs based on erroneous specs
135
Copyright 2013 by Data Blueprint
Relative Cost/Effort to Repair System in Relation to Development Stage
(Adapted from [Davis 1990.)
$0.00
$2.00
$4.00
$6.00
$8.00
$10.00
$12.00
$14.00
$16.00
$18.00
$20.00
Coding UnitTest
AcceptanceTest
Maintenance
Nearly 50% of problems are detected only after
completion of acceptance tests
Requirements
Design
136
Copyright 2013 by Data Blueprint
Evolving Data is Different than Creating New Systems
137
Common Organizational Data (and corresponding data needs requirements)
New Organizational Capabilities
Systems Development
Activities
Create
Evolve
Future State
(Version +1)
Data evolution is separate from, external to, and precedes system development life cycle activities!
Results
Increasing scope and depth of information architecture utility
Individual SDLC Effort
Copyright 2013 by Data Blueprint
Individual SDLC efforts make increasing use of IA
• Over time the:– Number of requests increase– Utility of the results increase– Amount of metadata contributed by
new systems development increases
138
Requirements
Design
Implement
Requests
Results
Individual SDLC Effort
Requirements
Design
Implement
Requests
Results
Individual SDLC Effort
Requirements
Design
Implement
Requests
Organized system metadata
Organized system metadata
Organized system metadata
Copyright 2013 by Data Blueprint
Data is not a Project• Durable asset
– An asset that has a usable life more than one year
• Reasonable project deliverables – 90 day increments– Data evolution is measured in years
• Data– Evolves - it is not created– Significantly more stable
• Readymade data architectural components– Prerequisite to agile development
• Only alternative is to create additional data siloes!
139
• A bit of history- Clinger Cohen Act - how's it going?
• Motivations: - Poor data management performance to date
➡ (requires additional or different effort) - Recognition that data is not a project
➡ (requires a different approach)- Lack of domain expertise
➡ (requires different career preparation)• The role of a CDO - three necessary but insufficient
prerequisites: 1. Dedicated solely to data asset leveraging2. Unconstrained by an IT project mindset3. Reporting directly to the business
Copyright 2013 by Data Blueprint
The Case for the Chief Data OfficerRecasting the C-Suite to Leverage your most Valuable Asset
140
The Top Job• Finance• Operations• Sale/Marketing• HR• Risk• Technology/CIO
– Align IT initiatives with business goals– Improving IT operations performance– Cultivating the IT/business partnership– Cost control/expense management– Implementing new systems– Leading change efforts– Driving business innovation– Redesigning business processes– Developing and refining business strategy– Negotiating with IT vendors– Managing IT crises– Developing market strategies & technologies– Security management– Studying trends to identify opportunities
Copyright 2013 by Data Blueprint
Where does data go?
141... data
Information
Copyright 2013 by Data Blueprint
Reporting Particulars1. Report outside of IT
and the current CIO altogether;
2. Report to the same organizational structure that the CFO and other "top" jobs report into; and
3. Focus on activities that are outside of (and more importantly) upstream from any system development lifecycle activities (SDLC).
142
Top Operations
Job
Copyright 2013 by Data Blueprint
CDO Reporting
143
Top Job
Top Finance Job
Top InformationTechnology
Job
Top Marketing
Job
• There is enough work to justify the function• There is not much talent• The CDO provides significant input to the Top Information Technology Job
Data Governance Organization
ChiefData
Officer
Copyright 2013 by Data Blueprint
Comments from CIOs• I think the need for a newly defined executive level role of
data/information ownership is clear, and is distinct from the role of leading the IT function. However, if you propose changing the CIO’s title to something that is much more appropriate for the vast majority of CIO’s, like CTO, and reusing the CIO title for the new function you will meet with a firestorm of dissent. That dissent will not be in opposition to your core suggestion that each organization needs to establish this new function in the business. That need can be clearly articulated and proven. The dissent will be around moving today’s CIO in that direction, or, worse, stripping them of that “honorable” title. That will be a debate full of sound and fury but signifying nothing, other than distracting everyone from the core argument at hand - - the need for the creation of that role in the organization.
144
Copyright 2013 by Data Blueprint
Reporting Particulars1. Report outside of IT and
the current CIO altogether;2. Report to the same
organizational structure that the CFO and other "top" jobs report into; and
3. Focus on activities that are outside of (and more importantly) upstream from any system development lifecycle activities (SDLC).
145
Copyright 2013 by Data Blueprint
1. IT does not feel the true impact of poor DM practices– Data problems can and often do delay IT projects and this causes
the organization to pay more for IT services than it should. However the impact to the business of unmanaged/unmanageable data is greater than on IT. The business is more likely to suffer publicly from data problems while the impact of IT-related data failures is less likely to become know outside of the organization.
2. IT does not that they do not know organizational business rules that govern data and its use. – IT's focus on technical knowledge leaves little resources and few cycles to devote to
understanding how data is used by the business for various decisions.
3. IT does not own or control access to the subject matter expertise ultimately needed to implement business driven data centric data development practices. – Again, the focus on technical implementation details, keeps IT fully occupied with "how" concerns.
There is very little time for understanding business oriented, "what" problems.
4. Only the business can competently assign values on various data uses. – Defining what “good enough” for data management practices cannot be done from an IT perspective.
5. Reporting to IT has been expected because IT owned the method. – Since IT specializes in methods, IT was assumed to be the function to also manage the data. Most
non-IT folks have no idea that methods can be used to develop flexible, adaptable, and reusable data.
6. CIOs are already slammed. – Since CIOs are generally unsuccessful at remaining data-focused it is best to create a position solely
dedicated to just that function.
146
Arguments for not reporting to IT
1. Appropriate division of labor. • Since IT has its hand full with the how's of the
systems, it is appropriate that the business take ownership of defining the organizational "what."
2. Reduction in organizational system requirements translations. • The current organizational cycle begins with the
business defining system requirements to IT who codifies these and relates them back to the business for verification.
3. Encourages business to learn appropriate IT-derived methods. • Maintaining the requirements (the "whats") in the business will reduce
translations and encourage broadly based, information systems-oriented thinking.
4. Better encourages data and architectural component reuse. • The longer-term focus of data is – at odds with IT development cycle –
DM is a program not a project.
Copyright 2013 by Data Blueprint
Arguments for reporting to the business
147
Copyright 2013 by Data Blueprint
148
New division of labor• Reporting to IT
– Data Development– Database Operations
Management• Shared with the
business– Metadata
Management– Data Security
Management• Reporting to
Business– Data Architecture
Management– Reference & Master
Data Management– Data Warehousing &
BI Management– Document & Content
Management– Data Quality
Management– Data Governance
Copyright 2013 by Data Blueprint
Separating the Wheat from the Chaff
149
Copyright 2013 by Data Blueprint
Separating the Wheat from the Chaff• Harvard Business School wisdom
– 20% of your customers cause 80% of your problems– Eliminating those 20% (the problem customers) should
increase profitability without any additional cost– The key question is how to identify the correct 20%– Very few try this strategy
• Poor data management practices are costing organizations much money/time/effort
• Data that is better organized increases in value• Pareto analysis dictates that 80% of organizational data is ROT
– Redundant– Obsolete– Trivial
• The question is the same - which data to eliminate?
150
Copyright 2013 by Data Blueprint
What to do first?1. Providing effective and efficient
data management to the organization. • Understanding the current and future data needs of
an enterprise and making that data effective and efficient in supporting business activities.
2. Separating the data into ROT and non-ROT. • 80% of organizational data is ROT: Redundant;
Obsolete; or Trivial. Moving to reduce organizational data ROT significantly reduces the complexity of the organizational data management challenge.
3. Remaining data focused. • The TDJ maintains a sole focus on understanding, marshaling, and applying
organizational data resources in support of business strategies. Laser like focus on data will help the organization to understand how data fuels various organizational processes and permits meaningful l enhancements to organizational data inventory. Any time the TDJ is paying attention to anything other than data support for strategy, they are diluting their own effectiveness.
4. Individuals possessing experience in this area will commend a premium as their specialized knowledge skills, and abilities will be in demand and experienced individuals will be in short supply
151
Copyright 2013 by Data Blueprint
CDO Survey: What key qualifications are necessary to be a successful CDO?
152
88.5% 83.2%
66.4% 62.8% 61.1% 59.3% 59.3%
55.8% 51.3%
46.0% 44.3% 41.6%
38.9%
29.2%
0.0%$
50.0%$
100.0%$
Experience in operationalizing
Data Governance,
Data Stewardship and
Data Quality
Strong leadership and C-suite/board
communication skills
Expertise in creating and
deploying best practices and
methodologies
Experience in defining business
requirements for information
management projects
Extensive industry
knowledge, including
expertise at the intersection of risk and other
domains
Experience in leading major information
management programs in key business areas related to the
industry
Information management program life
cycle experience
Familiarity with Enterprise Metadata
Management (business and IT) and OMG
standards
Expertise in creating and
leading best of class business
and IT teams for information
management
Familiarity with process
modeling, semantic
modeling and data modeling
Familiarity in setting up and
supporting information
analytics teams
Familiarity with industry data
models
Expertise in business and IT
architecture, including
familiarity with leading
architectural standards such as TOGAF, FEA and/or Zachman
End-to-end data warehousing
program execution
knowledge and leadership
93.8% 85.8% 82.3% 82.3%
74.3% 72.6% 66.4%
54.9%
45.1% 38.9%
29.2%
0.0%$
50.0%$
100.0%$
Possess a balance of
technical skills, business
knowledge and people skills
Outstanding relationship building and
communication skills
Politically savvy
Leader and visionary
Team builder Understands privacy, data security, and
risk management
components of data
Understand core
information domains of
risk, industry knowledge,
product/service and customer
Can work with pure IT and information
management specialists to
bridge the information gap
SME in requisite
business side as well as
methodologies and practices
needed to effectively connect business
requirements to IT
Not too technical but
not a technical novice
Entrepreneurial
93.8% 85.8% 82.3% 82.3%
74.3% 72.6% 66.4%
54.9%
45.1% 38.9%
29.2%
0.0%$
50.0%$
100.0%$
Possess a balance of
technical skills, business
knowledge and people skills
Outstanding relationship building and
communication skills
Politically savvy
Leader and visionary
Team builder Understands privacy, data security, and
risk management
components of data
Understand core
information domains of
risk, industry knowledge,
product/service and customer
Can work with pure IT and information
management specialists to
bridge the information gap
SME in requisite
business side as well as
methodologies and practices
needed to effectively connect business
requirements to IT
Not too technical but
not a technical novice
Entrepreneurial
Copyright 2013 by Data Blueprint
CDO Survey: What key traits are necessary to be a successful CDO?
153
93.8% 85.8% 82.3% 82.3%
74.3% 72.6% 66.4%
54.9%
45.1% 38.9%
29.2%
0.0%$
50.0%$
100.0%$
Possess a balance of
technical skills, business
knowledge and people skills
Outstanding relationship building and
communication skills
Politically savvy
Leader and visionary
Team builder Understands privacy, data security, and
risk management
components of data
Understand core
information domains of
risk, industry knowledge,
product/service and customer
Can work with pure IT and information
management specialists to
bridge the information gap
SME in requisite
business side as well as
methodologies and practices
needed to effectively connect business
requirements to IT
Not too technical but
not a technical novice
Entrepreneurial
Copyright 2013 by Data Blueprint
73% of CDO Functions Are Less that 1 Year Old
• Does your CDO have a budget?
154
• Does your CDO have a staff?
Copyright 2013 by Data Blueprint
Chief Electrification Officer
155
• Chief Electrification Officer – responsible for electrical generating and distribution systems. The title was used mainly in developed countries from the 1880s to 1940s during the electrification of industry, but is still used in some developing countries.
Not all roles are needed always!
Copyright 2013 by Data Blueprint
Propositions• A rather serious gap exists
in most organizations• The TDJ should be a
business function• Any time the TDJ is paying attention
to anything other than data supportfor strategy, they are diluting their own effectiveness
• The TDJ is be a business leader who is the CTO's primary customer; requiring new, not-widely available KSAs
• New titles are required for most existing CIOs • Most current CIOs should have their job titles changed to CTOs• The vast majority of organizations lack qualified personnel and/
or organizational focus on improving the process by which organizational strategy is informed by organizational data resources.
156
Copyright 2013 by Data Blueprint
CDO Success1. Dedicated solely to data asset
leveraging
2. Unconstrained by an IT project mindset
3. Reporting to the business
157
10124 W. Broad Street, Suite CGlen Allen, Virginia 23060804.521.4056