Using the PI Integrators to Improve the Value of your PI data

2016 REGIONAL SEMINARS

Martin Bryant, Field Service

September 20, 2016

Using PI Integrators to Improve the Value of Your PI System Data


Seeking Value in a Sea of Buzz and Jargon

Process Efficiency

Internet of Things

Big Data

Data Mining

Advanced Analytics

Business Intelligence

Operational Intelligence

Advanced Visualization

Machine Learning

Predictive Analytics

Data Warehouse

Data Science

Data Lakes

2


The Importance of Data (and Sensor Data) is Increasing

3


“We’re looking toget the data into

tools like Spotfire"

“I need to be able to look at data

across similar anddifferent assets at

the same time”

“Writing custom code and

supporting itindefinitely is just

not an option”

Summary of Needs:

Familiar tools

Scalability without code

Flexibility and trust

OSIsoft has Listened to Your Needs

4


PI Integrators

PI System PI Integrators

PI Integrator for Business Analytics• SQL Server• Hadoop (HIVE & HDFS)• Oracle RDBMS• File• ODBC client

PI Integrator for SAP HANA• SAP HANA• Smart Data Integration (SDI)

PI Integrator for Microsoft Azure• SQL Data Warehouse• Azure SQL• Azure Data Lake

5

Time, Event and AssetContext

Tabular Context

PI Integrator forBusiness Analytics


The Traditional PI System


An Alternative Data Use Model


Solving complex problems for a fleet

Dashboarding – visual reporting – real time & mobile

Integration to new I.T. projects and databases

Multivariate, other statistics & machinelearning resourcesOne time answers or Running models

Drill drown, rollupAnywhere anytime

The right way to bring operational data to I.T.’s Big Data party…

26

Getting Value…


Why? Big data … complex problem solving

All PI Customers have a great deal

of data.. Often too much for

effective analysis in a spreadsheet.

This data can be complex with

many variables. Multivariable

analysis and machine learning can

provide interesting and invaluable

answers to complex, big

questions…

PI data is very large and complex.

This is about finding the answer to large, complex questions.

• Identifying patterns and discovering problems through statistical methods that require large and diverse datasets

Statistical Analytics

MultidimensionalBusiness Intelligence

& Dashboards

PredictiveStatistical Modelling& Machine Learning


Complex Analyses Increase the Need for Deeper Integration

Real-time visibility

Monitoring

Process Optimization

Benchmarking

System Optimization

• BI app (e.g. Tableau, Spotfire, Lumira)

• PI Integrator forBusiness Analytics

• PI Integrator for SAP HANA

• Machine learning (Azure ML, R)

• PI Integrator for Business Analytics


• Traditional HMI

Disparate assets Interacting with assets on an individual basis

Interacting with common assets as a fleet

Real-time & historical views across any

asset

Fleet-wide performance comparisons

Large scale multi-variate analysis

Co

mp

lexi

ty

• PI ProcessBook• PI Coresight• PI Datalink

10


Applying Data to Maintenance…

11

Wait for failure

faster break & fix

Monitoring

Performance

Predictive



Using run timesand not clock

hours to schedule

Preventive Maintenance

and inspections

Compare current performance

metrics, including vibration to

expected values

Based on the history of like

equipment and circumstances –

build a model that predicts failure and

triages maintenance

Co

mp

lexi

ty

Data aware


• What material is being hauled?

• When was it raining?• Are there holes in the

road?• What is the grade of

the hill?• When did breaks and

downtime occur?• How do driving

behaviors vary by shift?

• When did the geology change?

• Which well was being drilled?

• What angle was the drill bit?

• Is production related to drill conditions?

• Was wind gusty or steady?

• Was the maintenance planned?

• How long does this issue usually take to fix?

• What product is being made?

• When is the equipment empty?

• Where was the instrument when I took that measurement?

• How are renewables impacting the equipment?

• Was there a voltage violation?

• What are the changes in weather?

Mining Oil & Gas Wind Power Pharmaceuticals Transmission & Dist

Results and ROI: Data Integration can Address Key Questions

12


Why? Dashboarding

Don’t wait for tomorrow to explore performance today

your operational data is a click or swipe away.

Even from your mobile devices

• Visualizing diverse sets of data sources to gain insights, create reports, and improve operations

Visual Analytics

Scheduled and Ad-Hoc Reporting

• Automated daily reports

• Multiple data source views

• Management overviews

• KPIs and metrics

• Rollups by business unit

• Future Data (forecast)


Result: Improved, Detailed Reporting and Analytics

14

Dashboard drilldowns to detailed reports

Production Summaries

Drilling Activity Reports

RefineryKPI Scorecards

Regional Product

Inventories

Asset Performance Displays

PI Coresight Displays


Why? I.T. Integration projectsInformation systems have projects that provide enormous value –

and those can benefit from the real-time process awareness of PI

Data. But PI Data hasn’t been easy to integrate – until now.

Structured PI data in a format readily consumable by the latest I.T. tools.

• Centralizing data from different business systems

• More effectively analyzing and reporting on business and building LOB applications

Data Warehousing


OT IT

Infrastructure

New Technology is bringing the IT and OT Worlds Together

Plant Manager

Engineer CIO

Business Analyst

IT ArchitectI want to minimize risks through data

driven decisions

I want to compare my equipment

against our other sites

I want to build predictive models from historical data

COO

I want to spend less

time on operational reports I want trusted

production data, to be confident in our

decisions

I want all data accessible by the BI

tools my users already know

I want the operational data

to work with our other technologies

I want operational data

for the Big Data project we’re starting

I want to analyze production, maintenance logs, and financial

data all together

16


RenewablesEnergy production reportsWind farm comparisonsOutlier analysis

MiningRoute optimization Energy reduction Across 300 haul trucks

Food and BeverageUtility usagesProcess analytics

Life SciencesReactor comparisonsProcess scale-ups (1L, 3L, 10L, 1kL, 10kL)

Oil and GasDrilling and production comparisons Information distribution

PI Integrator for Business Analytics is in use today!

IT/ OT integration

Business intelligence and reporting

Data warehouse integration

Supporting cross-platform projects

17


(Big) Data Projects: Sound Attractive … But There are Challenges

64% of large enterprises plan to implement a big data project.85% will be unsuccessful.

Data cleansing and preparation tasks can take 50-80% of the development time and funds.

Source: https://hbr.org/2014/04/the-sexiest-job-of-the-21st-century-is-tedious-and-that-needs-to-change/

Succeed

Fail

Analysis

Clean & Prep

18

https://hbr.org/2014/04/the-sexiest-job-of-the-21st-century-is-tedious-and-that-needs-to-change/


(Big) Data Projects: Time Spent in the Wrong Areas

Source: http://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/#5481f6037f75

19

http://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/#5481f6037f75


Cleaning & Preparing Sensor Data: It’s Challenging

03:00 03:10 03:20 03:30 03:40 03:50

Speed

Oil

Temp

Oil

Temp

Speed

Torque

03:00

03:10

03:20

03:30

03:40

03:50

Oil

Te

mp

60

60

60

120

60

Oil

Te

mp

50

50

50

50

45

Sp

ee

d

10.0

10.2

9.8

10.0

10.1

Sp

ee

d

9.8

10

10

9.9

9.9

To

rqu

e

25

24

25

25

25

Comm Failure

Different start times

Additional Measure

Uneven Spacing

Spike / Out of Range

20


PI Integrators Let You Clean and Prepare PI System Data

for your business intelligence tools,data warehouses, and

data lakes

AssetsEvents

CleanseData quality

Augment Aggregation

ShapeModel

TransmitNormalization

Clean & Prepare

Analyze

21


PI Integrator for Business Analytics - “Select Data”

• Intuitive way to create tabular content in “PI Views”

• Requires AF Hierarchy

• Select AF Elements and Attributes

• Scale up leveraging name, hierarchy, or category

22


PI Integrator for Business Analytics - “Modify View”

• Select any time range and interval

• Add columns for proper aggregating PI System data

• Add columns for common time and date functions

23


PI Integrator for Business Analytics - “Publish”

• Select targeted endpoint “PI View”, MS SQL, text file, more to come….

• Publish once or on a scheduled bases

24


Let’s See This in Action!Publishing Building Energy Consumption

for 67 Buildings to SAP HANA

25


Operational Reporting & Analysis Architecture

27

… All BI tools that support ODBC

Tableau Spotfire

SAS

MSFT BI

PI Server

PI Integrator for Business Analytics – Business Intelligence Edition

System of Record

Visualization & Analytics

Data Preparation and Integration Layer

“I want to analyze operations data stored in the PI System using

modern BI tools!”

27


Enterprise Data Warehouse Architecture

28

PI Server

PI Integrator for Business

Analytics

Tableau Spotfire

System of Record



Oracle DW, SQL Server, TeradataEnterprise Data

Warehouse / Data Mart / Data Lake

Custom Applications

SASMSFT

BI

Custom or 3rd Party Data Management and ETL

EAMSalesCRM

ERP HR …

Hadoop

“I need to fit operational data into my existing

company IT information

architecture!”

28


PI Integrator for SAP HANA Architecture

SAP HANA

HANA SDA

PI Server

Data

Archive

PI Asset Framework (AF)

SAP DP Agent

HANA Studio

PI Integrator FrameworkWeb UI Shape Designer

Windows

Windows

Linux

Microsoft SQL

Linux / Windows (All Java) (via SDA)

SAP HANA IoT Connector by OSIsoftPI JDBC

PI View

Security REST Layer

Direct publication into a HANA schema

Option 1 Option 2

SAP Lumira, VORA, S4, etc.

Full platform coverage for all data integration scenarios!

29


Soon… PI Integrator for Azure SQL Datawarehouse

Azure SQL Data Warehouse

INGESTPREPAREDATA SOURCES

On Premise

Machine Learning

Power BI

ANALYZE PUBLISH CONSUME

Cortana

Web/LOB Dashboards

Azure SQL Data Warehouse

PI System

On Cloud

Azure Data Factory (Orchestration)

Predictions as Future Data (to PI 2015)

PI Integrator for Azure

30


Solving complex problems for a fleet

Dashboarding – visual reporting – real time & mobile

Integration to new I.T. projects and databases

Multivariate, other statistics & machinelearning resourcesOne time answers or Running models

Drill drown, rollupAnywhere anytime

The right way to bring operational data to I.T.’s Big Data party…

26

Getting Value…


Move the Needle with PI Integrators

PI Integrator for Business AnalyticsPI Integrator for SAP HANAPI Integrator for Microsoft Azure

• Start the conversation!• Could a colleague make a better

decision with data you see daily?

• What business intelligence tools could you leverage further?

• Visit YouTube or osisoft.com to see which PI Integrator works for you

32


Contact Information

Martin Bryant

[email protected]

Field Service Engineer

OSIsoft, LLC

3333

mailto:[email protected]


Questions

Please wait for the microphonebefore asking your questions

Please remember to…

Complete the Surveyfor this session

State your name & company

34


Thank You

2016 REGIONAL SEMINARS42

The PI Integrator for Business Analytics fully realizes the PI infrastructure goal by providing peerless real-time & deep historical PI process data to enterprise “big data” dash-boarding, integration, analysis and data scientist tools with easy configuration and performance.

PI Integrator for Business Analytics


PI Integrator For Business Analytics

• No programming or queries - browser based configuration

• Integrated to PI AF - uses templates as schemas

• Rest Model Accessed with ODBC driver

• New fast PI ODBC Driver & PI Data Access Server

• Windows Integrated security with delegation (Kerberos)

• Updates for new data on a schedule



Integrator Targets:

• Microsoft SQLServer

• Hadoop

• Oracle

• Ascii files/CSVs

• SAP Hana (from SAP)

• Microsoft SQL Datawarehouse for Azure (cloud based)



• Supports PI Event Frames

• Time Sampling analogous to PI Datalink

• Data cleansing (nulls/errors), row filtering (by value, etc..),

data typing (string/numeric) for each column

• Aggregations supported – calculate averages, totals for

intervals.


OSIsoft University ProgramPI Integrator For Business Analytics

Compared to previous OSIsoft data access technologies:

• Much easier to learn (2 hrs instead of 1 day - configured not coded)

no coding – no code generators

• Much higher performing (many millions of rows instead of <100K)

~ 25K outbound events per second

• Integrates to key 21st century technologies

endorsed by major tech players and many large companies

More secure

2016 REGIONAL SEMINARS 53

What’s new?

Is this merely a new product built by OSIsoft? or does it fit into a larger perspective?

Relational database technology is changing

Technology is changing – new opportunities from “Big Data” and internet of things


SQL – the end of an era

• Tablular. Not object oriented

• File oriented. Not scalable

• Security Challenges

SQL and relational / tabular technologies are not adequate to today’s data.



Web Services – not adequate for big data

• Too slow for big data

• Transmission and query – but no storage

• Programming – but no client tools

• Security Challenges



Best of both worlds technologies required:

REST (representational state transfer) technologies

• Web interface and transport – designed for the internet• Performance – designed for memory resident scalability • Standard data formats including binary formats• Object oriented• Secure in the current cross platform environment



Impact of Social Media

Impact of Big Data / Internet of Things Technology

Google, Facebook, and other social media sites have generated massive amounts of data.. This may not seem immediately relevant to industrial applications, But it has driven new search engine and database technologies.. And it has resulted in self-service clients to do analysis


Impact of Internet of Things

Impact of Big Data / Internet of Things Technology

As computing becomes less expensive…Every “thing” gets a computer – refrigerators, fitness wristbands, cars, drones, mobile devices,

and new industrial devices too These devices are not organized and concentrated pyramidalyand that is demanding and providing newsecure data communications and database technology.


The Big Data Landscape

59

From: http://www.bigdatalandscape.com/

http://www.bigdatalandscape.com/


Why? Big data … complex problem solving

All PI Customers have a great deal of

data.. Often too much for effective

analysis in a spreadsheet. This data

can be complex with many variables.

Multivariable analysis and machine

learning can provide interesting and

invaluable answers to complex, big

questions…

PI data is very large and complex.

This is about finding the answer to large, complex questions.

• Identifying patterns and discovering problems through statistical methods that require large and diverse datasets


• How are renewables impacting the grid?

• What kind of grid components are associated with voltage violations?

• What is the impact of changing weather?


Complex Analyses Increase the Need for Deeper Integration

61

Real-time visibility

Monitoring

Process Optimization

Benchmarking

System Optimization

• BI app (e.g. Tableau, Spotfire, Lumira)

• PI Integrator forBusiness Analytics


• Machine learning (Azure ML, R)

• PI Integrator for Business Analytics


• Traditional HMI



Real-time & historical views across any

asset

Fleet-wide performance comparisons

Large scale multi-variate analysis

Co

mp

lexi

ty

• PI ProcessBook• PI Coresight• PI Datalink


Enabling Analytics for Operational IntelligenceReal-Time Decision Analysis Retrospective & Predictive Analysis

DescriptiveCondition & Performance


& Dashboards

Time and EventTrending & Awareness


Tabular Context



Specialized ModelsSimulation & Optimization


Why? I.T. Integration projectsInformation systems have projects that provide enormous

value – and those can benefit from the real-time process

awareness of PI Data. But PI Data hasn’t been easy to

integrate – until now.

Structured PI data in a format readily consumable by the latest I.T. tools.

• Centralizing data from different business systems

• More effectively analyzing and reporting on business and building LOB applications

Data Warehousing


OT IT

Infrastructure

New Technology is bringing the IT and OT Worlds Together

Plant Manager

Engineer CIO

Business Analyst

IT ArchitectI want to minimize risks through data

driven decisions

I want to compare my equipment

against our other sites

I want to build predictive models from historical data

COO

I want to spend less

time on operational reports I want trusted

production data, to be confident in our

decisions

I want all data accessible by the BI

tools my users already know

I want the operational data

to work with our other technologies

I want operational data

for the Big Data project we’re starting

I want to analyze production, maintenance logs, and financial

data all together

17


No single purpose data pools – data lakes that are:

• Reusable & Flexible

• Vetted & Maintained

• Secure

• Integrated with data from other sources

65


Why? Dashboarding

Industrial companies are replacing periodic row and column reports (day,

shift, etc..) with dynamic web-based dashboards – and they are exploring

them from their phones and mobile devices. They are combining the near

real time operational data with financial and administrative data for the

comprehensive view of the immediate road ahead.

Don’t wait for tomorrow to explore performance today

your operational data is a click or swipe away.

Even from your mobile devices

• Visualizing diverse sets of data sources to gain insights, create reports, and improve operations

Visual Analytics


Use Case: Management Summaries

67









Result: Improved, Detailed Reporting and Analytics

68

Dashboard drilldowns to detailed reports

Production Summaries

Drilling Activity Reports

RefineryKPI Scorecards

Regional Product

Inventories

Asset Performance Displays

PI Coresight Displays


Big Data Projects Sound Easy… But there are Complications!

85%

64% of large enterprises plan to implement a big data project.

85% will be unsuccessful.

Data cleansing and preparation tasks can take 50-80% of the development time and funds

https://hbr.org/2014/04/the-sexiest-job-of-the-21st-century-is-tedious-and-that-needs-to-change/

Analysis

Prep

10


“We’re looking toget the data into

tools like Spotfire!"

“I need to be able to look at data

across similar anddifferent assets at the same time!”

“Writing custom code and

supporting itindefinitely is just

not an option!

Summary of Needs:

Familiar tools

Scalability without code

Flexibility and trust

OSIsoft has Listened to Your Needs

12


OSIsoft’s Solution Lets You Prepare and Deliver Process Data

71

HARMONY

CLEANSE

AUGMENT SHAPE

TRANSMIT

PULL

PUSH

to any Visualization Tool or Analysis database

via the ODBC standard


Operational Reporting & Analysis Architecture

72

72

… All BI tools that support ODBC

Tableau Spotfire

SAS

MSFT BI

PI Server

PI Integrator for Business Analytics – Business Intelligence Edition

System of Record



“I want to analyze operations data stored in the PI System using

modern BI tools!”


Enterprise Data Warehouse Architecture

73

73

PI Server

PI Integrator for Business

Analytics

Tableau Spotfire

System of Record



Oracle DW, SQL Server, TeradataEnterprise Data

Warehouse / Data Mart / Data Lake

Custom Applications

SASMSFT

BI

Custom or 3rd Party Data Management and ETL

EAMSalesCRM

ERP HR …

Hadoop

“I need to fit operational data into my existing

company IT information

architecture!”


Tips - Meta Data & Hierarchy

• You can’t roll up and you can’t drill down without hierarchy

• And Meta-data (installation date, mode, long/lat) is where the slicers for great BI comes from


Tips - JoinsUsing this technolody – Joins are done in the client

We had pretty clear naming of our elements –f you are having problems with joining to unique objects – two hints

One to many joins – it’s easier to join if one database has only one record per unique item (one per turbine or transformer or substation – you can use this as an index and join other tables to it and they’ll transfer those relationships through it.

If your naming is not very unique – you can insert the ElementID in as an attribute and join on that. We did that in this example even though we didn’t need to.


Tips - Categories

If you have multiple tables coming from the same element – you can predefine what you want to put in each table by marking them with categories… Sort by category in the integrator and pull the whole group into your attribute / column selection.


Tips - Aggregates

The default method for getting data into a column for the integrator is interpolationThe fastest method is “last recorded value”But often the best is an average ..

Don’t be shy about the average


Takeaways

As always – developments in the larger technology world (Big Data, the Internet of Things) are producing innovations in database and communications technologyAt OSIsoft – we are not just aware of this – we are embracing it…With technologies that use new standards and methods but areconsistent with your decades ofhistory…

26



• No programming or queries (based on PI AF)

• Writes to appropriate Big Data databases (Hadoop, SAP)

• Cleanse, Filter, Shape your data – and it will update

• Secure

Compared to previous OSIsoft data access technologies:

• Much easier to learn and use

• Much higher performing

Takeaways


Use Cases

• Production Reporting– More detailed view into energy, oil, metals

• Alerting and Customer Intimacy– Integrate detailed production data with CRM data to alert on outages

and meet customized SLA requirements

• Regulatory Compliance– Keep product genealogy data on hand for products to deal with

regulatory requests

• Root Cause Analysis– Discover patterns related to equipment failure or low quality product

82


PI Integrator for SAP HANA

SAP HANA Platform Edition

SAP IoT Connector(Smart Data Access)

Lumira

Vora

S4

Etc.

HANA Database

SAP HANA Cloud Platform

Smart Data Streaming

Full platform coverage for all data integration scenarios

Data Pulls(Federated)

Data Access Pattern

Product / Feature

Roadmap

Databases and Applications

83


PI System

Released: PI Integrator for Business Analytics & SAP Hana

PI View

Visual Analytics

PI ODBC


Publish

Pull

84


Enabling Analytics for Operational IntelligenceReal-Time Decision Analysis Retrospective & Predictive Analysis

DescriptiveCondition & Performance


& Dashboards

Time and EventTrending & Awareness


Tabular Context



Specialized ModelsSimulation & Optimization

86


Use Case: Management Summaries

87








Using the PI Integrators to Improve the Value of your PI data

Documents

Transcript of Using the PI Integrators to Improve the Value of your PI data