The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… ·...
Transcript of The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… ·...
![Page 1: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/1.jpg)
The Rules of Subsurface AnalyticsJane McConnell, Practice Partner Oil and Gas, Teradata
DEJ KL, 4 October 2017
![Page 2: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/2.jpg)
2
• Why subsurface analytics is different
• The Rules– Rule 1: Right People
– Rule 2: Right Platform
– Rule 3: “Good Enough” Data Management
– Rule 4: Agile Approach
– Rule 5: Business Buy-in
• Recap
Agenda
© 2016 Teradata
![Page 3: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/3.jpg)
3
Why Subsurface Analytics is different
© 2016 Teradata
![Page 4: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/4.jpg)
4
1940’s business computing:where it all started for business analytics
![Page 5: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/5.jpg)
5
But for us, the IT/OT divide happened
IT and “the business” diverge on business critical solutions
IT failed to keep up with competitive computing
developments
Business units and teams become fertile ground for
point solutions…
…which become silos…
…which become app suites that only subject matter experts understand and
manage
…so IT never get near it beyond providing power and
network bandwidth
![Page 6: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/6.jpg)
6
Custodianshipv
Stewardship
Custodian
• Controls access
• Avoids unknown data
• Transfer data
• Avoids risk
• Hates change
• Acquires knowledge
• Creates walls
Steward
• Shares data
• Celebrates variety
• Enables access
• Embraces risk
• Owns change
• Shares insights
• Teaches governance
Subsurface Data Management Culture
![Page 7: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/7.jpg)
7
Rule 1: Right People
© 2016 Teradata
![Page 8: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/8.jpg)
8
• Too many disciplines for any one person to know it all
• “T-shaped people” who go wide across many disciplines but deep into their specific domain
• Need outstanding data management and data engineering skills (and culture)
• Need platform expertise for sustainability and deployment
• Need Subject Matter Expertise
Right People, plural. And T-shaped.
© 2016 Teradata
![Page 9: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/9.jpg)
9
Ingest Curate Analyse Visualise
Analytics / data science workflow
footer
© 2017 Teradata
9
![Page 10: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/10.jpg)
10
Here’s one we did earlier in Malaysia
• Working closely with the customer – Subject matter expertise
– Source system expertise
• Teradata Subject Matter Expert
• Data management skills
• Data platform skills
• Coding skills
• Data science skills
• Frontend/visualisation skills
![Page 11: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/11.jpg)
11
Rule 2: Right Platform
© 2016 Teradata
![Page 12: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/12.jpg)
12
• Kerry Blinston, CGG, ECIM 2014
• “Knowledge development” applications come with import filters for specific file types and specific tasks
• Data is modelled logically for well-defined (and hence brittle) processes that may not reflect all (or even any!) use cases
• Only “perfect” data can be imported into applications or schemas
© 2014 Teradata
The problems with existing data stores
New data types, or new combinations, challenge all of this
![Page 13: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/13.jpg)
13
• If we don’t give them a purpose-built platform for analytics, we will be in Desktop/Excel Hell.
• A platform that – Accepts data from any
discipline
– Makes it easy for data scientists to use their tools – R, Python etc
– Provides the right level of governance and data quality
• Call it a Data Lake, or call it an Analytical Data Platform– Just build something where they
can get data and ideally write new data back
Build a new platform that all disciplines can use
© 2016 Teradata
![Page 14: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/14.jpg)
14
Rule 3: Good enough Data Management
© 2016 Teradata
![Page 15: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/15.jpg)
15
• Stewards mentor and support “citizen data management”
• Everyone cares about the data and its quality
• Everyone can do something about it when they find bad data
• Data governance is a function of data value
“Good Enough” means:
• Good: don’t compromise on quality
• Enough: don’t boil the ocean
“Good Enough” Data Management
![Page 16: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/16.jpg)
16
Don’t be a data hoarder!
Why not store data at a granularity good enough to extract value?
• Granular enough
• Dimensioned (time, space) enough
• Resample, interpolate, aggregate
• Profile data so you can get an idea of it quickly
© 2014 Teradata
What should data look like?
![Page 17: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/17.jpg)
17
• Parse out the measurement data
• Link it through time and space
• Relate using metadata and master data
• Form a view on whether a hypothesis is worth developing
© 2014 Teradata
“Difficult file formats” (Multi-structured data)
![Page 18: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/18.jpg)
18
Text
• Language
• Typos
• Consistency
• Quality
Use simple characterisation tools to understand what is in the data
Don’t try to build a whole text input and cleansing framework
© 2014 Teradata
Dealing with unstructured data
![Page 19: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/19.jpg)
19
Rule 4: Agile Approach
© 2016 Teradata
![Page 20: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/20.jpg)
20
Agile, Scrum, DevOps, Interactive Visualisation
© 2016 Teradata
![Page 21: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/21.jpg)
21
What not to do
Common resource-sinks:
• Point solutions
• Technology projects
• Waterfalls
• Brittle data modelling
• “We want ML/AI!”
© xkcd.com
![Page 22: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/22.jpg)
22 © 2017 Teradata22
Iterate.One project at a time.Deliver value often.
Prepare
Contextualise and plan
Form problem statement
Prioritise by impact
Communicate analysis plan and responsibilities
Prepare
Contextualise and plan
Form problem statement
Prioritise by impact
Communicate analysis plan and responsibilities
Execute
Build on prior work
Validate data
Recheck hypotheses
Drive insights and recommendations
Execute
Build on prior work
Validate data
Recheck hypotheses
Drive insights and recommendations
Review
Create concise business story
Highlight overall business impact
Include assumptions and sources
Follow up with business on the actions
Review
Create concise business story
Highlight overall business impact
Include assumptions and sources
Follow up with business on the actions
Deliver
Post code to repository
Stakeholder contacts
Final presentations
Deliver
Post code to repository
Stakeholder contacts
Final presentations
Assimilate
Store well-commented SQL
Document in wiki
Train BU in tool usage
Assimilate
Store well-commented SQL
Document in wiki
Train BU in tool usage
![Page 23: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/23.jpg)
23
Working together
Willing to take risks
Proactive
Speed / Time-to-insight
Agility needs the right mindset
© 2017 Teradata
![Page 24: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/24.jpg)
24
Rule 5: Business Buy-in
© 2016 Teradata
![Page 25: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/25.jpg)
25
Data management to enable business-aligned data science
© 2017 Teradata25
• What financial, operational or
environmental impact are you
delivering?
▪ What techniques, functions,
workflows and skills are required?
▪ What data is required and in
what form?Data Management
Data Science
Business
Value
![Page 26: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/26.jpg)
26
Business-focused data management
• Embed data ownership in the business units
• Engage with business leadership to plan, budget and deliver data-driven initiatives
• Define and drive data exploitation strategy
• Understand data value and leverage high value data for business impact
…do we need a
Chief Data Officer?
Source: Andrew White, Gartner 2017
![Page 27: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/27.jpg)
27
Recap
© 2016 Teradata
![Page 28: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/28.jpg)
28
The 5 Rules – all equally important
© 2016 Teradata
Right People Right Platform Good Enough Data Management
Agile Approach
Business Buy-in
![Page 29: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/29.jpg)
29
Stated another way - Data Management 2.0
© 2017 Teradata29
Stop doing
• Brittle data
management
• Silos
• Disposable data
science
• Transfer and analysis in
Excel
Keep doing
• Applying domain
expertise
• High levels of
governance
• Driving data quality
• Learning
Start doing
• Aligning with business
• Applying context
• Data profiling
• Enriching data
• Applying critical
thinking
![Page 30: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/30.jpg)
30
Closing thoughts“Trying to plan for the future without
a sense of history is like trying to
plant cut flowers.”
Daniel J Boorstin (1914 – 2004)
12th Librarian of the US Congress 1975-1987
“Trying to plan for the future without
a sense of history is like trying to
plant cut flowers.”
Daniel J Boorstin (1914 – 2004)
12th Librarian of the US Congress 1975-1987
Jane McConnell
Practice Partner O&G , Industrial IoT Group
Thing Big Analytics, a Teradata Company
+47 98282110
My blog on Forbes
My blog on Teradata.com
Follow me on Twitter @jane_mcconnell
My profile
![Page 31: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/31.jpg)
3131
© 2016 Teradata
Jane McConnell
Practice Partner O&G , Industrial IoT Group
Thing Big Analytics, a Teradata Company
+47 98282110
My blog on Forbes
My blog on Teradata.com
Follow me on Twitter @jane_mcconnell
My profile
![Page 32: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/32.jpg)
32
What is the point of data science?.
Data science, also known as data-driven science, is an interdisciplinary field
about scientific methods, processes, and systems to extract knowledge or insights
from data in various forms, either structured or unstructured, similar to data
mining.
Wikipedia, 2017
![Page 33: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/33.jpg)
33
Ingest Curate Analyse Visualise
Data science workflow
footer
© 2017 Teradata
3
3
![Page 34: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/34.jpg)
34
• Finding relationships with complex data sets
• Characterising behaviour and understanding the demographics of data
• It can be applied to:
–Data profiling and QC
–Data preparation
–Data mining
–Operational processes
–Data art
Not just machine learning
![Page 35: The Rules of Subsurface Analytics6a1c78bd5aa99a799e4a-994bd2c9c91957858d6a6a567c1584c0.r8.… · Data science, also known as data-driven science, is an interdisciplinary field about](https://reader034.fdocuments.us/reader034/viewer/2022042417/5f331c966ae7861ced46975e/html5/thumbnails/35.jpg)
35 footer© 2017 Teradata35
How other industries grew an analytics culture…
Prepare
Contextualise and plan
Form problem statement
Prioritise by impact
Communicate analysis plan and responsibilities
Prepare
Contextualise and plan
Form problem statement
Prioritise by impact
Communicate analysis plan and responsibilities
Execute
Build on prior work
Validate data
Recheck hypotheses
Drive insights and recommendations
Execute
Build on prior work
Validate data
Recheck hypotheses
Drive insights and recommendations
Review
Create concise business story
Highlight overall business impact
Include assumptions and sources
Follow up with business on the actions
Review
Create concise business story
Highlight overall business impact
Include assumptions and sources
Follow up with business on the actions
Document
Post code to repository
Stakeholder contacts
Final presentations
Document
Post code to repository
Stakeholder contacts
Final presentations
Assimilate
Store well-commented SQL
Document in wiki
Train BU in tool usage
Assimilate
Store well-commented SQL
Document in wiki
Train BU in tool usage