Innovations in Grid Computing with Oracle Coherence
-
Upload
bob-rhubart -
Category
Technology
-
view
3.313 -
download
1
description
Transcript of Innovations in Grid Computing with Oracle Coherence
1 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Innovations in Grid Computing with Oracle Coherence Craig Blitz Oracle Coherence Product Management
2 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Thefollowingisintendedtooutlinegeneralproductuseanddirec4on.Itisintendedforinforma4onpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunc4onality,andshouldnotberelieduponinmakingpurchasingdecisions.
Thedevelopment,release,and4mingofanyfeaturesorfunc4onalitydescribedforOracle’sproductsremainsatthesolediscre4onofOracle.
3 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Presentation Objectives To Understand…
1)WhatisdrivingtheneedforDistributedCachingandIn‐MemoryDataGridproducts?
2) HowCoherenceimprovesperformance,enableslinearscalability,andprovidesavailabilityforsharedservices
3)Coherence’sStrategicDirec8onandLatestFeatures
4 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Internet Scale
Problem:Howtodealwiththefloodofcustomerdemandforservicesthatoccurswhenyouexposeyourinfrastructuretoalarge,poten4allyunboundednetwork.
‐Or,statedmoreposi4vely‐
Opportunity:Howtobuildinfrastructurethatallowsyoutoscalebusinessandimprovemarginsbyexposingyourservicestoalarge,poten4allyunboundednetwork.
5 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
The Internet Scale Problem is Multi-Faceted
• Virtually unlimited number of customers • Services exposed to partners’ overuse • Customers’ experience relies on partners’ services • Global and Real-time
6 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Scaling Infrastructure to Meet the Problem
• Applications must act on data in real-time – Access, query, and aggregate the data – Trigger business actions based on changes to data
• Must scale predictably and cost-effectively – Failure to do so results in loss of revenue and cost overruns – No time to re-architect in Internet time
• Need a new paradigm in data management
7 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Oracle Coherence
• Elastic caching is “the secret sauce for internet scale” • In-Memory Data Grid pioneer since 2001 • Gartner’s Leader’s Quadrant for Enterprise Application Servers,
2009
8 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
More. Faster. Cheaper.
• Scaletomoreusersandmoredata
• Scalepredictablyandcost‐effec4vely
• Actondatainreal‐4me
• Reducespendingonbackendsystems GetontheGrid
9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Oracle Coherence
• Coherence: in-memory data grid – Distributed caching to applications – Scalable extreme transaction
processing – Real-time eventing, query, and map/
reduce aggregations – Abstraction from back-end data
sources – High-availability to applications
EndUsers
Applica4onServers
Mainframes,Databases,etc.
Applica4onObjects
10 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Oracle Fusion Middleware Complete, Open, Integrated, Best-in-Class
DevelopmentTools
UserEngagement
SocialWeb Mobile
EnterpriseManagement
Iden4tyManagement
ContentManagement
BusinessIntelligence
BusinessProcessManagement
ServiceIntegra4on
DataIntegra4on
CloudApplica4onFounda4on
11 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Presentation Objectives To Understand…
1)WhatisdrivingtheneedforDistributedCachingandIn‐MemoryDataGridproducts?
2) HowCoherenceimprovesperformance,enableslinearscalability,andprovidesavailabilityforsharedservices
12 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Enterprise Application Scalability Challenges
ServiceClientsBrowsers,supplierandpartnerclients,applica4onclients,mobileapps,andotherserviceconsumers.
ServiceImplementa8onCustomapplica4ons,BPMprocesses,servicebusendpoints,UIservices,andotherserviceproviders.
EaseofScaling
SharedServicesRDBMS,cloudservices,supplierandpartnerservices,mainframeapplica4ons,andsoon.
• Growsandscalesnaturally
• Increasingload• Computa4onallyintensivework
• Highlydependentonsharedservices
• Complexandexpensivetoscale
• Highlatency,SPOB,andSPOF
13 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Traditional Clustering
• Dependent on shared service latency
• Increase in size of cluster leads to increase in load on shared service
ServiceNode1
SharedService
ServiceNode2
ServiceNodeN
CachedObjects
CachedObjects
CachedObjects
ServiceClients
• Inconsistent view of data across cluster nodes
• No possibility for in-memory analytics
• Limited in-memory cache due to • Capacity implications • Java garbage collection
14 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Enter the Coherence Data Grid
• Clustered Caching – Consistent view of data
across cluster • Partitioned Caching
– Increase in size of cluster adds capacity but does not decrease performance
• Distributed Data Processing – Enables a scalable platform
for high performance in-memory analytics
ServiceNode1
EaseofScaling
ServiceClients
SharedServices
In‐memoryDataGrid
ServiceNode2
ServiceNodeN
15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
How to Integrate with Coherence Two Options
Custom integration through Coherence API
- or –
Through Existing ‘Switch-On’ OOTB Integrations
Oracle TopLink Grid: JPA object-relational data Oracle Coherence*Web: HTTP sessions Other Oracle Products that integrate Coherence
16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Coherence Basic Inteface Custom Integration
• Application uses Coherence API to store/access its data
• Application cache properties are configured externally through XML file
• Basic cache access through NamedCache Map interface
NamedCache neCache = CacheFactory
.getCache("NetworkElement");
…
NetworkElement ne = new NetworkElement(“ID-321”, “OC-196”, “Nortel Networks);
neCache.put(“ID-321”, ne);
…
NetworkElement ne = neCache.get(“ID-321”);
17 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Management Partitioned Caching
• Extreme Scalability: Automatically, dynamically and transparently partitions the data set across the members of the grid.
• Pros: – Linear scalability of data capacity – Processing power scales with data
capacity. – Fixed cost per data access
• Cons: – Cost Per Access: High percentage
chance that each data access will go across the wire.
• Primary Use: • Large in-memory storage environments • Parallel processing environments
18 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Management Partitioned Fault Tolerance
• Automatically, dynamically and transparently manages the fault tolerance of your data.
• Backups are guaranteed to be on a separate physical machine as the primary.
• Backup responsibilities for one node’s data is shared amongst the other nodes in the grid.
19 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Management Near Caching
• Extreme Scalability and Extreme Performance: The best of both worlds between the Replicated and Partitioned topologies. Most recently/frequently used data is stored locally.
• Pros: – All of the same Pros as the Partitioned
topology plus… – High percentage chance data is local to
request. • Cons:
– Cost Per Update: There is a cost associated with each update to a piece of data that is stored locally on other nodes.
• Primary Use: – Large in-memory storage environments
with likelihood of repetitive data access.
20 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Source Integration Read Through / Write Through
• Read Through – All data reads occur through cache – If there is a cache miss, the cache will load the data from the data
source automatically
• Write Through – All data writes occur through cache – Updates to the cache are written synchronously to the data
source
21 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Source Integration Write Behind
• All data writes occur through cache • Updates to the cache are written asynchronously to the
data source
22 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Processing Parallel Query
• Programmatic query mechanism • Queries performed in parallel across the grid • Standard indexes provided out-of-the-box and supports implementing
your own custom indexes
23 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Processing Continuous Query Cache • Automatically, transparently and dynamically maintains a view locally
based on a specific criteria (i.e. Filter) • Same API as all other Coherence caches
24 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Processing Invocable Map
• The inverse of caching • Sends the processing to where the data is in the grid
– Once and only once guarantees – Processing is automatically fault-tolerant
• Processing can be: • Targeted to a specific key or collection of keys • Targeted to any object that matches a specific criteria (i.e. Filter)
25 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Data Processing Events
• Build Complex Event-Driven Apps • Java Bean Model • Key-Based • Filter-Based
26 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Presentation Objectives To Understand…
1)WhatisdrivingtheneedforDistributedCachingandIn‐MemoryDataGridproducts?
2) HowCoherenceimprovesperformance,enableslinearscalability,andprovidesavailabilityforsharedservices
3)Coherence’sStrategicDirec8onandLatestFeatures
27 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Release Overview
• Oracle Coherence 3.7.1 – Released September 23 – Strategic investment in:
• Exalogic Innovation • Ease of Use • Integration Points
– Significant hardening
© 2011 Oracle Corporation
28 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8 © 2011 Oracle Corporation
"OracleCoherencehasbeenamarketleaderhelpingcustomersscaleapplica4onstomeetthedemandsofinternet‐scalecompu4ngforoveradecadenow.Coherence3.7.1con4nuestoreducecomplexity,easeintegra4on,andaccelerate4metomarketofscalablesolu4ons...Atthesame4me,whenrunningonanOracleExalogicengineeredsystem,Coherence3.7.1leveragesOracle'sExabustechnologytoachievethelowestlatenciesandhighestthroughput.”
CameronPurdyVPDevelopment,Oracle
29 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Elastic Data Virtual Memory for Your Data
• Revolutionizes the scale of data grids • Use block storage for cached data
– Optimized for flash – Works well with NAS, Disk, etc.
• Simplifies capacity planning and deployment – Configure amount of on-heap data to store – Overflow data written to block storage – Data stored in buffers until flushed to journal – Reduces chances of Out-Of-Memory errors
CacheServer
1
Heap
Flash
CacheServer
2
Heap
Flash
CacheServer
3
Heap
Flash
30 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
REST Support
• REST API – cache creation – data access & modification – parallel aggregation/
processing/querying • Easy Integration
– iOS, Ruby, PHP, Python… – Via the HTTP protocol
• Compliments C++/.NET/Java Support
© 2011 Oracle Corporation
31 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Coherence API Types
MapAPIReal‐TimeClientAPI
QueryFilterAPI
Aggrega4on(Map‐Reduce)
API
GridProcessingAPI
ScalableDistributedCachetoScalable,Event‐DrivenReal‐TimeApplica4ons
CoherenceCluster
CoherenceCluster
WANReplica4on
PerMemberCacheStore/LoaderAPI
DataSource
Java/C++/.NET
REST (Python, PHP,…)
32 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Query Explain Plan
• Evaluate query cost and index effectiveness
• Quicker time to market of optimized Coherence solutions
• Explain Plan • Provides the estimated cost of
evaluating a filter as part of a query operation
• Trace • Performs the associated query • Provides the actual cost of
evaluating a filter as part of a query operation.
© 2011 Oracle Corporation
33 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Live Objects New Coherence Incubator Project
• Building blocks for an event-driven finite-state machine – …that is distributed – …and scalable – …and resistant to machine failure
• Architectural possibilities: – Stated Event Driven Processing – Ripple Effect
34 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Coherence on Exalogic Engineered System Optimized Scalability and Performance in a Box
• Coherence optimized for Exabus • Pre-configured network/compute nodes • Elastic Data: Expand Capacity with Flash • Easy deployment as demand spikes • Grow from ¼ to multi-rack
4x Throughput, 1/6th Latency!!!
35 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
GoldenGate/Coherence Integration
• Simplifies Coherence use in shared database environment – Propagates DB updates to Coherence – GoldenGate captures changes to
database tables – TopLink maps database changes to
cached objects – Availability—2012 – GoldenGate 11gR2 – Available as patch to WebLogic 12.1.1
and 11.1.1.6/10.3.6 – Oracle database only
Database
CoherenceEnabled
Applica4on
Coherence
Read/WriteThrough
CustomPackagedApplica4on
DirectD
atabaseUpdates
UpdateCoherence
UpdateCoherence
GoldenGateJavaClient
TopLinkGridCoherence
CacheUpdater
DatabaseLogStream
Capture
<entity class="Address"> <table name="ADDR”/> <attributes> <id name="id"> <generated-value /> </id> <version name="ver“/> </attributes> </entity>
JPAMetadata
36 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
CoherenceIntegra8ons
Meta‐dataServices(MDS)
Real‐8meDecision
SOAInfrastructure
OracleADFOracleGlassFish
OracleAccessManager
OracleDataIntegra8on
TopLink
WebLogicPortal
OracleServiceBus
ServiceDeliveryPlaQorm
WebLogicServer
BIPublisher
OraclePeopleSoR
WebCenter
OracleBEAM
ATG
OracleCEP
OracleEn8tlementServices
OracleSecureTokenService
OracleIGBU
OracleCGBU
ATGHostedServicesDataCacheSimpleClustering
DataGridAdvancedCapabili4es
OracleIden8tyFedera8on
OracleWebServicesManager
SOATechnologyAdapters
SOABusinessRules
SOAHumanWorkflow
SOABPEL
SOACoherenceAdapter
OracleGoldenGate
37 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Oracle Enterprise Pack for Eclipse (OEPE)
• Coherence Project Configuration – Facets – Library Management – Descriptor Generation
• Runtime Configuration – Launch Config Editor – Run/Deploy/Debug from IDE
• Configuration Support – tangosol-coherence-override.xml editor – Validation – Context Sensitive Help Integration – Cache Configuration Editor (OEPE 1.8)
© 2011 Oracle Corporation
38 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
For More Information
• General Information: http://coherence.oracle.com • Coherence Training: http://education.oracle.com • Coherence Discussion Forum: http://forums.oracle.com • Coherence User Group on Linkedin • “Oracle Coherence 3.5” by Aleks Seovic • My email: Email: [email protected]
39 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
Q & (hopefully) A
40 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
PerformanceMonitoring&Diagnos4cs
Administra4on
Configura4onManagement Provisioning
Oracle Enterprise Manager 12C Comprehensive Coherence Management
41 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
• Completelycustomizableperformanceviews–savedifferentviewsfordifferentmonitoringrequirementsoftheenterprise
• Proac4veAlertsandno4fica4onsbasedongranularthresholds• Topologyviewtoshowassocia4onsanddependencies• Logmonitoring–generateAlertsbasedonlogpajers• Advancedmonitoring–pushreplica4on,reapsessions,transac4onalcaches,etc
DeepPerformanceVisibility&AlertsBasedProac4veMonitoring
• Real‐4methreadsanalysis–findcallstack,locks,methodlocalvariables,etc• Real‐4meheapanalysis–heapdistribu4onindifferentspaces(eden,perm‐gen,etc),garbagecollec4on
• Differen4alheapanalysis–pinpointheapleaksbycomparingtwosnapshots
Real‐4meJVMDiagnos4csforCoherenceNodes
Oracle Enterprise Manager 12C Monitoring & Diagnostics
42 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
• Querybaseddataopera4ons• CentralUIforkeycacheopera4ons–view,update,import,export,purge,add/removeindexes
• Dras4callysaves4meincacheopera4onsandmakesiteasyforadministrators
• Savequeriesforfuturereferences
CentralizedCacheDataManagement
• Maintain‘goldimage’inthesolwarelibrary• Provisioningnewclusteroraddnodestoanexis4ngcluster
• SupportUnicastAddrressaswellasWKAbasedclusters
AutomatedProvisioning
Oracle Enterprise Manager 12C Administration & Provisioning