A Journey to the Center of the Clouds...PROTOTYPE TEST PO and team representatives Research...

1A Journey to the Center of the Clouds© 2018 Carnegie Mellon University

SATURN 201814th Annual SEI Architecture Technology User Network Conference

MAY 7–10, 2018 | PLANO, TEXAS

A Journey to the Center of the CloudsJørn Ølmheim


SATURN 2018

Who am I?

• Polyglot programmer• Practicing architect• Open source enthusiast• Amateur guitar player• Overall geeky dude• … and so much more …


SATURN 2018

Agenda

• What is the cloud?

• Statoil’s cloud gamble

• The Cloud Journey – Architecture

• Pilot project architecture

• Migrating an existing application

• 3rd party cloud integration

• Enterprise Data Modelling

• A new generation of apps

• Learnings


SATURN 2018


SATURN 2018

Cloud basicsJust the short version.


SATURN 2018

Cloud basics

Cloud Characteristics:Self-service

Network access

Multi-tenancy

Elasticity

Measured

Service Models:Infrastructure as a Service (IaaS)

Platform as a Service (PaaS)

Software as a Service (SaaS)

Read more here >>

http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf


SATURN 2018

Our cloud strategy

Statoil has a Cloud First strategy. All new IT solutions, software or computing capabilities (includingchanges to existing ones) shall be evaluated to be a cloud service.

Not "We must use cloud", but "We always consider cloud and checkif it benefits Statoil"

From“The master of IT Operation & Maintenance”

To“The master of Data & IT Service consumption”


SATURN 2018

Cloud benefits

Start-up

•No CAPEX•Less deployment time

Operation

•Scalability & Pay per use

•Collaboration with partners

•Multiplatform / mobility included

•Upgrades included

Exit

•Exit by ending subscription

•No hardware involved

SpeedAgility

Improved utilisation of IT personnel and assets


SATURN 2018

Cloud risksInformation

security

Contracts / Terms and conditions

«Shadow IT»

Exit serviceData:• Personal information• Export control• Geodata

Customisation


A Journey to the Center of the Clouds

Statoil’s bold cloud gamble


SATURN 2018

Exp D&W Corporate Vessel Refine-

ries TradingO&MPetec

From Silos of Data…..


SATURN 2018

Exp D&W Corporate Vessel Refine-

ries TradingO&MPetec

From Silos of Data….. to one common Data Platform for the Statoil Value Chain

Statoil Data Platform Orchestrate all Statoil Data

From Silos of Data…..


SATURN 2018

SSDLThe Vision

Enable Statoil subsurface to become a fully data-driven company. A data-driven company is an organisation where every person can use data to make better decisions. All personnel have access to the data they need when they need it.

Find and produce more profitable oil and gas by using the Data Lake to combine knowledge and experience with data and interpretations to unlock potential, improve understanding and create value

• A system to access all data in existing subsurface discipline silos

• Make all data available for the user to run analysis on• Something beyond a Hadoop based storage platform

Geosciences

Supply chain

From TSB2025D – Subsurface Data Lake


SATURN 2018

Statoil Data Platform Orchestrate all Statoil Data

Cloud Platforms

Petec: Landmark

Drilling: Kongsberg

ERP: SAP

Equipment Monitoring: GE Predix / SiemensExp: Schlumberger

Maj

or P

latf

orm

Par

tner

s

From Silos of Data….. to one common Data Platform for the Statoil Value Chain



The Cloud Journey - Architecture


500 GB every day - per turbine

150.000 readings - per secondHistorical data - for 40 years


1 GB every 10. second32 TB per survey

Every 3. year => 2x per year

SchlumbergerHaliburton

…..Data platform

Statoil Data Platform – Overall platform approach

IIoT,

Pla

nt &

R

ealti

me

Dat

a

External Connection

Advanced Analytics

Business SpecificSystem


LDWMobileWeb Bots


Data Streaming

Statoil Current IT - On premise & IaaS

Data Virtualization

Data Platform Storage

Data replicationAPI

Statoil Data Platform

Sources:

• DBR• SAP• Subsurface • datamarts• Endur• ++++++

Automated Batch Loading

Business SpecificSystem Software as a Service, in the cloud


SAP…..

Data Platform

E,P,C,IProjects

Data Plaform

Portals

APIAPIAPIAPI

FileSystem

Parse / replicate



Pilot project architecture


OPENWORKS EDM LANDMARK WellDB

DSIS

Data lake

/raw/corporate/WellDBEDM LANDMARKOPENWORKS

/transform/corporate/

/data/subsurface/

WellDB

dataplatform-data

Subsurface data API

CSD DBR

CSD DBR

CompositeLog

FormationPressure

CPILog

DepthFormationPicks

CompletionIntervals

CasingDepths

Wellpaths

CompletionData

Dashboards and web applications

RECALL

FMU

UNIX file share

Fault lines

Outline

Maps structure

Master data API

FieldWellWellboreStratigraphy

OPENWORKS EDM LANDMARK

ASC

LASDLIS

PerforationData

CompletionString

Perforationdata

WellOperations

CasingDepths

PerforationDepths



Migrating existing applications


Cloud migration short term


SATURN 2018

External SaaS integration


SATURN 2018

Cloud migration long term



3rd party cloud integration


SATURN 2018

Cloud future vendor landscape


SATURN 2018

3rd party SaaS Integration



Enterprise Data Modelling


Some data challenges

• No common overview of available data assets.

• Data in multiple sources (< 2500 internal systems), but no common insight into which data sits where.

• Data needed across domains, but difficult to know who to contact and to combine data across different data ecosystems.

• Lack of common terms and definitions of cross-domain key data objects.

• Users spend more time looking for data, than they do analyzing it.

• Users are busy re-producing data that already exists.


Share and integrate

Enterprise Data Management (EDM) toolboxTo find, understand, share, use and govern data assets

Describe and organize Govern and implement

• Business Information Inventory (BII)

• Data definitions in Business Glossary

• Data and information models

• Statoil Data Catalog

• Enterprise Data Model

• Metadata model

• Roles and responsibilities

• Organization and resources

• Processes and policies

Data vocabulary

• International standards


Business glossary

Business object

=========

=========

=========

=========

=========

Enterprise Data Model


Enterprise Data Model v1


SATURN 2018

Domain Driven Design – Strategic Design

Context Map

Continous Integration

UbiquitousLanguage

SharedKernel

PublishedLanguage

AnticorruptionLayer

Customer/SupplierTeams

Conformist

Open HostService

Bounded Context

SeparateWays

CoreDomain

Domain VisionStatement

GenericSubdomains

SegregatedCore

HighlightedCore

AbstractCore


Strategic design: Context maps

In large systems (or set of systems), we need a map to give us a picture of the models that are inside.

Bounded context

Bounded context

Context map

Relations



A new generation of apps


SATURN 2018

TESTPROTOTYPEIDEATEEMPATHIZE DEFINE

PO and team representatives

Research (preferablyonsite)

Define the coreissue to be solved

PO and team representatives

Iterate over MVP prototypewith actual business user

UX Lead - Team

Actual business user

Team input and experiments

Alignment on way forward

Interactive prototype

Solves core issue?

DESIGN THINKING


SATURN 2018

Learnings

• Getting data out of the legacy systems is not as difficult as we initially expected• Some legacy formats can provide a challenge

• The real challenge is to agree on the apis and data models• Existing models mostly dictated by vendors• Application specific vs Discipline specific vs Cross-discipline models

• Bottom-up vs Top-down design of apis and storage

• Introduce DDD concepts to enterprise data modellers

• Building apps to solve specific real user needs using Design Thinking



Thank you!

A Journey to the Center of the Clouds...PROTOTYPE TEST PO and team representatives Research...

Documents

Transcript of A Journey to the Center of the Clouds...PROTOTYPE TEST PO and team representatives Research...