Technology in support of utilities challenges
-
Upload
aitor-ibanez -
Category
Technology
-
view
1.196 -
download
4
description
Transcript of Technology in support of utilities challenges
<Insert Picture Here>
Technology for Utilities Transformation and new Challenges: Integration, Big Data, Extreme Performance.Aitor IbañezEnterprise Architect for Utilities
2
<Insert Picture Here>
Contents
• Utilities Business Transformation• Need for Technology• Extreme Performance• No-limit Integration• Applications: Architectural Challenges
3
<Insert Picture Here>
Utilities Business Transformation
4
Data Growth defines new Challenges…
4
DATA GROWTH IMPACT
• Multiple Devices: Meters, Apps, SCADAs,…
• Info Processed by multiple Departments: Risk Mgmt., Billing, Demand Analysis, Network.
• Growth of Corporate DW for Reports and Trend Analysis.
• Agility for Business Processes on Transactions between Devices and Applications
• New Scalable Architecture supporting new Analysis and Transaction Growth Reqs.
• New Data Mgmt Reqs. for Storage, Master Data, Synchronization & Sharing of Information
Fuente: Accenture “Challenges for Utilities” 2010, Oracle Insight
IT CHALLENGES
5
Growing Market Share
Defense against Competency
Cost Savings
Growing Sales
Managing efficiently growing Data Volumes
Improving Customer Care & Go-to-market
Processes
Simplifying Storage and Servers
On-demand growing, Business Continuity
Business Processes Automation
Scalable Infrastructure
More Power
Improving Performance and
security of Transactions
Reducing Operational Costs and improving
Energy Efficiency
Faster processing to improve Billing,
Demand Analysis, …
5
Improving AvailabilityReal-time Data Access, Load
balancing
External Pressure Strategy
Business Perspective
Levers Implications
Ma
na
gin
g D
ata
Ex
plo
sio
n
wit
ho
ut
co
st
inc
rem
en
ts
IT Perspective
Challenges that arise new Biz & IT Needs
Utilities need New Technology to face up New Challenges
6
… Needs translated into new Requirements
• Scalable Infrastructure• Horizontal growing
• Business Processes Automation• Faster deployment of processes• Process Governance• Transactions
• Real-time Data Access, Load Balancing• Fast response to data needs• Grid computing
• On-demand growing, Business continuity• Right-sizing, growing infrastructure as the Business demands• SLAs Compliance
• Simplifying Storage and Servers• Reducing HW Installed base: best platform for every purpose• Energy Savings
• More Power• Unprecedented Processing Power• Driving new Business opportunities
6
Performance
Scalability
Manageability
Protecting Investment
7
<Insert Picture Here>
Need for Technology
8
Smart Grid ChallengesManaging the Exponential Growth of Data Volumes
New devices in the homeenabled by the smart meter
You are here.
AMI Deployment
PCTs Come On-line
Distribution Management Rollout
Mobile Data Goes Live
RTU Upgrade
GIS System Deployment
OMS Upgrade
Advanced Distribution Automation
Substation Automation System
Workforce Management Project
Time
200 TB
400 TB
600 TB
800 TB
Expected Growth of a Tier 1 Utility’s Data Volume
New devices in the homeenabled by the smart meter
You are here.
AMI Deployment
PCTs Come On-line
Distribution Management Rollout
Mobile Data Goes Live
RTU Upgrade
GIS System Deployment
OMS Upgrade
Advanced Distribution Automation
Substation Automation System
Workforce Management Project
Time
200 TB
400 TB
600 TB
800 TB
New devices in the homeenabled by the smart meter
You are here.
AMI Deployment
PCTs Come On-line
Distribution Management Rollout
Mobile Data Goes Live
RTU Upgrade
GIS System Deployment
OMS Upgrade
Advanced Distribution Automation
Substation Automation System
Workforce Management Project
Time
200 TB
400 TB
600 TB
800 TB
Expected Growth of a Tier 1 Utility’s Data Volume
Source: EPRI
Much more data… Much more processing…
Processing Power required for Interval based billing
ScalableSystem
Break Point System
Co
st
Meters
Costinviable
Scalability break points: storage, scalability cost
9
Integration beyond BoundariesResponding to events and information from a variety of sources
10
So… Need for Technology!
• What do we need?
Extreme Performance
Massive Data Volume
Unstructured Data Analysis
No-Limit Integration
Biz Processes Automation
Services Architecture
Engineered Systems
Events Processing
Data Integration
BPM, SOA, EDA
11
<Insert Picture Here>
Extreme Performance
12
Exadata Database Machine
Database Grid Intelligent Storage Grid
InfiniBand Network
• Redundant 40Gb/s switches• Unified server & storage network
• 14 High-performance low-cost storage servers
• 8 Dual-processor x64 database servers
OR
• 2 Eight-processor x64 database servers
• Scalable Grid of industry standard servers for Compute and Storage • Eliminates long-standing tradeoff between Scalability, Availability,
Cost
• 100 TB High Speed disk, or336 TB High Capacity disk
•5.3 TB PCI Flash
• Data mirrored across storage servers
Copyright © 2010, Oracle Corporation and/or its affiliates – 12 –
13© 2008 Oracle Corporation – Proprietary and Confidential – 13 –
Traditional Vs Smart Scan Processing
I/Os Executed:
1 terabyte of data returned to hosts
DB Host reduces
terabyte of data to 1000 customer names that are returned to client
Rows Returned
SELECT
customer_nameFROM calls
WHERE amount > 200;
Table
Extents Identified
I/Os Issued
2MB of data
returned to server
Rows Returned
Smart Scan
Constructed And Sent To Cells
Smart Scan
identifies rows and columns within
terabyte table that match request
Consolidated
Result Set Built From All
Cells
SELECT
customer_nameFROM calls
WHERE amount > 200;
Traditional Scan processing Exadata Smart Scan processing
14
Exadata Storage Software Unique Features
• Exadata Smart Scans• 10X or greater reduction in data sent
to database servers
• Exadata Storage Indexes• Eliminate unnecessary I/Os
• Hybrid Columnar Compression• Efficient compression increases
effective storage capacity and increases user data scan bandwidths by a factor of up to 10X
• Exadata Smart Flash Cache• Breaks random I/O bottleneck by
increasing IOPs by up to 20X• Doubles user data scan bandwidths
• I/O Resource Manager (IORM) • Enables storage grid by prioritizing
I/Os to ensure predictable performance
• Quality of Service (QoS)• Actively meet and maintain SLAs• Memory Guard to protect existing
current transactions from memory-based failures
Copyright © 2010, Oracle Corporation and/or its affiliates – 14 –
15
Proof of Concepts Results
0
500
1000
1500
"01" "07" "OF " Queries
Referencia “As IS” Exadata Optimizacion
47 veces más rápido
68 veces más rápido
60 veces más rápido
2 veces más rápido
16
Oracle Exalogic Elastic CloudApplication Grid, Integrated Compute, I/O, Networking and Storage
• Central storage for system images
• Clustered for HA• 40 TB SAS disk• 4 TB read cache• 72 GB write cache
Integrated Storage
• 40 Gb/sec links• 10 Gigabit Ethernet
connectivity to datacenter
I/O Fabric
EL X2-2
• 30 x86 compute nodes• 360 Xeon cores (2.93 GHz)• 2.8 TB DRAM• 960 GB SSD
X2-2
Copyright © 2011 Oracle Corporation - Proprietary and Confidential
EL X2-2
Coherence
Enterprise M
anager Oracle Linux and/or Solaris
Exalogic Elastic Cloud Software
JRockit and HotSpot
WebLogic Server
Exalogic Elastic Cloud Hardware
Tuxedo
17
Oracle Exalogic and Exadata TogetherDesigned for Seamless Interoperability, Unmatched Performance
Exalogic Exadata
960 Gigabits/second(Maximum: 24x InfiniBand QDR)
• Exclusive: direct InfiniBand integration!
• Extreme performance, reliability, security
• Simple to deploy and manage
Active GridLink for RAC
• Resource-aware load balancing
• Instantaneous connection failover
• Transaction affinity to RAC nodes
• SQLnet optimized for InfiniBand SDP
Up to
3X OLTP Performance
Copyright © 2011 Oracle Corporation - Proprietary and Confidential
18
Technologies in SupportExaMDM Concept: Technical Architecture
10 M Meters Benchmark• ½ Exadata + ½ Exalogic + Oracle MDM + CC&B
• 500.000 bills 1 hour• Daily meter reads upload 2,6 hours• VEE processing 4 hours• Usage Aggregations 30 min.• Daily Total: 8 hours.
•Scale: ½ Full ExaMDM = 4 hours
19
Oracle Exalytics – Under the Hood
For information purposes only, not a commitment to the development, release, and timing of any features or functionality.
In-Memory Analytics Software
Essbase
TimesTen for Exalytics
Adaptive In-Memory Tools
1 TB RAM40 Processing Cores
High Speed Networking
In-Memory Analytics HardwareOracle Business Intelligence Foundation Suite
InfiniBand
Speed-of-thought enterprise Business Intelligence platform
Extreme performance via Exadata
20
Oracle ExalyticsExtreme Performance for Enterprise Analytics
• Speed-of-Thought Analysis at Enterprise Scale• 20x better response times• Sub-second performance enables speed-of-thought analytics• Grow to 50,000 enterprise users on one appliance
• What-if Analysis and Forecasting• 16x better response times • More accurate – use more detail, include more scope, evaluate wider operations• More scale – more users and higher volumes at busy periods• Greater agility – do daily instead of monthly
• Enterprise Planning• Reduce planning cycle times with 5x better response time and 6x better throughput• Run more planning models with finer grained operational detail to improve accuracy• Extend beyond Finance to every LOB, with 10,000 Planning users on one
appliance
21
Big Data for Smart GridAnalysis from unstructured data
• Unstructured data coming from• Smart Meters & Devices, Domotic,…• Info coming from Network & Generation SCADAs• Substations, other equipments,…• Meteo Data.
• BigData Appliance• 18 nodes, 216 CPU cores.• 864 GB RAM, 648 TB storage• 40 Gb/s Infiniband
• Extreme Performance for Unstructured Data processing.
22
Big Data for Smart GridThink about the Future…
• Driving Immediate Analysis with unprecedented processing power• Social Networks: mentions [ourCompany]
• Sentimental Analysis, Complaints• Social Networks: mentions #Meter, [ourCompany], #power Trending
Topics.• Targeted programs for each consumer
• Understanding Customer: CRM, Meter load, Internet logs, Twitter,…
• Power Demand Planning: processing trends and actual consumption by geography/concentrator.
• Prediction of Network failures or bottlenecks.• Geographical & Meteo Data
23
Divided Solution Spectrum
Acquire AnalyzeOrganize
MapReduceSolutions
DBMS (DW)
DBMS (OLTP)
Advanced Analytics
DistributedFile Systems
Transaction (Key-Value)
Stores
ETL
NoSQL Flexible
SpecializedDeveloper
Centric
SQL TrustedSecure
Administered
DataVariety
InformationDensity
Unstructured
Schema
24
DataVariety
InformationDensity
Unstructured
Schema
Oracle’s Big Data Solution
Acquire AnalyzeOrganize
OracleDatabase
(DW)
OracleDatabase
(OLTP)
In-DBAnalytics
“R”Mining
TextGraphSpatial
OracleBI EE
Oracle NoSQL DB
HDFS Hadoop
OracleData Integrator
Oracle Loader for Hadoop
Big Data Appliance• Hadoop
• NoSQL Database
• Oracle Loader for hadoop
• Oracle Data Integrator
Oracle Exadata• OLTP & DW
• Data Mining & Oracle R
• Semantics
• Spatial
Exalytics• Speed of
ThoughtAnalytics
25
Smart Metering Architecture DecisionsBig Data
Business Intelligence
meter Head-End
CIS
Work & Asset
Mobile
Distribution management
Outage Management
Self Service / portal
MDM
TBs
SOA Framework
Middleware
& Bus. Orchestration
SOA Framework
Middleware
& Bus. Orchestration
AM
I S
ervi
ces
AM
I S
ervi
cesNRT
26
<Insert Picture Here>
No-Limit Integration
27
Oracle GoldenGateThe Solution for Enterprise-wide Real Time Data Needs
Real-time information
Oracle GoldenGate delivers real-time access of real-time information, enabling companies to dramatically improve the availability, reliability, and performance
of critical data across enterprise systems.
Database and applications, Mixed sources, distributed systems, legacy,
OLTP, OLAP
Mission Critical Applications & Data, Business
Intelligence, Reporting for Customers, Partners & Employees
Real-time Access
28
Oracle Data Integrator
• E-LT Transformation• Declarative Design• No-intrusive• Change Data Capture• Architecture hot-
changes• Multi-DB support:
Knowledge modules
Data Integrator
Transactional
RDBMSCDC
Data Integration Solution
Logs
Realtime Data Hubs
Business Intelligence
Enterprise Warehouse
SOA | BAM | CEP
Oracle Data Integrator
Events
Services
E-LT
CDC
Logs
Rea
l Tim
e D
ata
Flo
w
Bat
ch D
ata
Flo
w
CDC
Logs
29
Supply Real Time Business IntelligenceNon-invasive Capture + E-LT Processing
Application Real Time BI(using Data Copy)
Analytic BI(Facts & Dims)
E-LT(Mini-Batch + Transforms)
Consistency Window
30
<Insert Picture Here>
Applications: Architectural Challenges
31
Technologies in Support Oracle Integration Platform for Utilities
Objective:Provide a comprehensive, consistent, and flexible integration approach across the Utilities Suite
Approach:• Built on Oracle’s Fusion Middleware • Leveraging AIA Methodology• Expand AIA Foundation Pack for Utilities:
–Common Objects and Enterprise Services–Compliant with utility industry standards–Deliver best-practice process models for key
processes across utility segments (water, gas, electric, waste)
Benefits:• Provide a common integration approach across
all Oracle applications. • Accelerate deployments and provide flexibility
for project-specific extensions.
Oracle Utilities Integrated Solution Suite-
SCADA
CRMML / AMIMWMWAM
MDMGIS / Spatial
ModelNMS
CC&B
QM
SOA Suite, AIA (ESB + FP for CIM and MultiSpeak)
Reference Process Model
Industry Standard
s
Common Object Model & Services
AIA Foundation Pack for Utilities
32
Oracle Utilities Integration StrategyOracle AIA Integrations Benefits
AIA Centers of Excellence
AIA Partner Co-development
UGBU Partners to certify with SAP + other 3rd Parties
Open EcosystemOpen Ecosystem
Pre-built integrations using leading Oracle Fusion middleware
Lower TCO
Supported products
Designed for next generation apps
compatibility
Reusability
ValueValue$$
Industry best practices
Prescriptive methodology
Industry standards adoption such as
CIM for Utilities
Proven by Oracle
MethodologyMethodology
Loose coupling of apps through SOA
Focus on configurations not
rebuilds
Successfully deployed in complex, high-
volume environments
ExtensibilityExtensibility
33
Business Process ManagementProcesses Life Cycle
Dashboards, tools and historical trends
Business Analyst
Process Developmentand Systems Integration
Process Developer
Business
Owner
Process Execution and Management
Users Interaction Business
ParticipantsCorporate
Applications
And Databases
Process Modeling, simulating and Documentation
34
Oracle Technologies in support Event-Driven Architecture
Complex Event Processor
BPM Processes
event
JMS
closed loop
Applications
event
Databases
Streams
event
event
Messaging
Event Sources
BAM Dashboards
BAM Alerts
BPM/BPEL Processes
Response Services
Coherence Real Time Listeners
event • Event sourcing
• Continuous query language to define event
• Correlation
• Sequencing
• Window of time evaluation
• Pattern matching
• Event publishing
• Real time Java infrastructure
35
Monitor business processes & services in real-time Key Performance Indicators (KPIs) Service-Level Agreements (SLAs)
Analyze events as they occur Correlate events & KPIs Identify trends as they emerge Alert users to bottlenecks & solutions
Act on current conditions Event-driven alerts Real-time dashboards BPEL processes & web services integration
Business Impact of Events, Oracle BAMMeaningful, Event-driven Intelligence for End-Users
36
Questions?