CON9528_PDF_9528_0001
-
Upload
markelshami -
Category
Documents
-
view
85 -
download
2
description
Transcript of CON9528_PDF_9528_0001
![Page 1: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/1.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1
![Page 2: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/2.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 2
Oracle Exalytics Best Practices -- Getting the Most out of Your Oracle Exalytics Deployment
Deb Bhattacharjee Director
Lionel Hsu Consulting Member of Technical Staff
Saugata Chowdhury Consulting Member of Technical Staff
Oct. 3, 2012
![Page 3: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/3.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 3
The following is intended to outline our general product direction. It is intended
for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
![Page 4: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/4.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 4
Agenda
What is Exalytics?
Exalytics Performance Tuning
Exalytics Considerations
![Page 5: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/5.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 5
WHAT IS EXALYTICS?
![Page 6: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/6.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 6
Oracle Exalytics In-Memory Machine
Benefits • Speed-of-Thought Interactive Visual Analysis
• Faster Planning Cycles with Richer Models
• Quick to Deploy, Supports More Users
Unique Features • In-memory Analytics
• Accelerates BI and EPM Apps
• Fits with existing data sources, infrastructure
• Full-stack optimizations – hardware & software
• Optimized with Exadata – InfiniBand
Oracle Exalytics is the industry's first in-memory machine that
delivers the fastest performance for business intelligence and planning applications
AVAILABLE TODAY
Oracle Exalytics is the industry's first in-memory machine that
delivers the fastest performance for business intelligence and planning applications
![Page 7: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/7.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 7
Exalytics Components
Memory Optimized
Hardware (1TB RAM, 40 cores)
BI Foundation Suite
Software (OS, middleware, BI)
In-Memory Software (TimesTen for Exalytics and
Optimized Essbase)
![Page 8: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/8.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 8
Oracle Exalytics Hardware
Memory 1 TB RAM Compute 4 Intel® Xeon® E7-4870, 40 cores total Networking 40 Gbps InfiniBand – 2 ports 10 Gbps Ethernet – 2 ports 1 Gbps Ethernet – 4 ports 8 Gbps FibreChannel – 2 ports Storage 3.6 TB HDD Capacity
![Page 9: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/9.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 9
Exalytics Storage Configuration
600 GB SAS - 0
600 GB SAS - 1
600 GB SAS - 2
600 GB SAS - 3
600 GB SAS - 4
600 GB SAS - 5
RAID 1
RAID 5 /u01(1.7TB) – software, TT data files
Single Disk Fault tolerant
Separate OS and Data disks
/boot (99MB) – boot partition
/ (447GB) – Operating system, logs, temp space
![Page 10: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/10.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 10
Exalytics Software Components
Timesten DB
Exalytics
Data Warehouse
Webcat
OBIPS
(sawserver) OBIS
(nqserver)
WLS (bi_server)
(analytics app)
Repository
Browser/Mobile Javahost
(Chart Engine)
Essbase
![Page 11: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/11.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 11
Exalytics – Some Performance Datapoints
• 2.7x throughput gain for on Exalytics when compared to 11.1.1.5 on same hardware (comparing peak throughput of 637 vs 235)
• Baseline’s throughput actually dropped under load due to lock contentions
Throughput Comparison - Baseline vs software optimization (datasource remaining constant) s
146
235
194
151121 127
71
205
368
493
563
605 619637
0
100
200
300
400
500
600
700
0 10 20 30 40 50 60 70 80
# of Virtual Users
Th
rou
gh
pu
t
Baseline - Throughput Exalytics - Throughput
![Page 12: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/12.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 12
Exalytics – Some Performance Datapoints
• BI Query reporting w/ Exalytics on Oracle DB
• In-memory aggregates (Adaptive In-Memory Cache) vs. the baseline DW: 18X faster on the average
• Individual dashboard response time gain upto 45x (Filter Department)
Response Time Comparison (Slowest transactions) - Baseline vs. Exalytics (non-Exadata)s
![Page 13: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/13.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 13
Exalytics – Some Performance Datapoints
• BI Query reporting w/ Exalytics on Exadata
• 23X average response time improvements thanks to in-memory aggregates, fast inter-connect and SQL optimizations for Exadata as compared to the base system
• Individual transaction gains upto 60X (Filter-Department)
Response Time Comparison (Slowest transactions) - Baseline vs. Exalytics w/ Exadata datawarehouses
![Page 14: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/14.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 14
Exalytics – Some Performance Datapoints
• About 5x response time improvement under load
• 7x improvement in CPU Utilization by Essbase
Response Time improvement for end-to-end scenario for Planning PSB usecase with Essbase s
0
100
200
300
400
500
600
20 40 60 80 100
AvgResponseTime insec (Exalytics)
Avg ResponseTime in sec(11.1.1.2.1)
![Page 15: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/15.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 15
Exalytics – Some Scalability Datapoints
• Up to 93x response time improvement with individual MDX Queries
• Up to 15x response time improvement for individual interactive calc scripts
• About 10x overall throughput improvement in mixed load scenarios accompanied by 1.8x CPU usage improvement
Response Time improvement for MDX Queries and interactive Calc under load with Essbase BSO Cubes
![Page 16: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/16.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 16
Exalytics
OBIEE Runtime Performance for dashboard navigation
– Less than 5% overhead for dashboard navigation
– Setup
Bare metal : 40 cores , 1 TB RAM
OVM ver 3.0.3: Dom 0 pinned to 10 cores, 40 vcpu for OBIEE VM and 40 vcpu for TT
Bare Metal Virtualized
(40 Cores) OBII 40 vcpu
TT 40 vcpu No of users (0 thinktime) 60 60
Throughput (tps) 337 329 <5% overhead in virtualized
environment with same core
count
Avg response time (sec) 0.18 0.18
Total CPU % (BI + TT) 91% 93%
Metrices Comments
Performance characteristics with OVM
![Page 17: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/17.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 17
Exalytics
Essbase BSO Interactive Calc and Query
– Less than 10 % overhead in virtualized environment compared to bare metal
– Setup
Bare metal : 40 cores , 1 TB RAM
OVM ver 3.0.3: Dom 0 pinned to 10 cores
9.1
16.3
22.6 25.5
8.4
15.7
21.7 22.9
0
5
10
15
20
25
30
1 2 3 4Th
rou
gh
pu
t T
xn
/se
c
# CPU Sockets and # of Essbase instances
Metal - 40 cores
OVM -- 40 cores
Performance characteristics with OVM
![Page 18: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/18.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 18
EXALYTICS PERFORMANCE TUNING
![Page 19: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/19.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 19
Timesten DB
Exalytics Performance Map • DB connection
•Log Level
Exalytics
Data Warehouse
Webcat
OBIPS
(sawserver) OBIS
(nqserver)
WLS (bi_server)
(analytics app)
Repository
Browser/Mobile
Javahost
(Chart Engine)
Essbase
•Hardware Acceleration flag
•Logical SQLs/Physical SQLs
•Init block
•Server Thread pool
• DB Gateway connection
pool
•OBIS Cache
•Temp/log folder location
•Log level
•JVM Heap Size for Trellis Charts
•Thread Pool size
•Javahost Clustering
•Temp/log folder location
•JVM Heap size
•Analytics app ConnectionPool
•Analytics app session timeout
•WLS LDAP connection pool
•Log folder location
•Hardware Acceleration flag
•Chart Thread Pool
•OBI PS Cache
•Session timeouts
•Cursor Wait Time
•Memory Quota
•Temp/log folder location
![Page 20: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/20.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 20
Exalytics Set the right Log Level • Set the default log level to ‘Error’
• Log levels should be the first place to check while debugging performance issues
![Page 21: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/21.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 21
Exalytics
Move temp files and log files to ramdisk for better scalability
– OBIPS
cd $BI_HOME/instances/instance1/tmp/OracleBIPresentationServicesComponent
mv coreapplication_obips1 coreapplication_obips1.localDisk
ln -s /dev/shm/coreapplication_obips1 .\
– Javahost
mkdir /dev/shm/coreapplication_obijh1
cd $BI_HOME/instances/instance1/tmp/OracleBIJavaHostComponent
mv coreapplication_obijh1 coreapplication_obijh1.localDisk
ln -s /dev/shm/coreapplication_obijh1 .\
– OBIS tmp folder
mkdir /dev/shm/coreapplication_obis1
cd $BI_HOME/instances/instance1/tmp/OracleBIServerComponent
mv coreapplication_obis1 coreapplication_obis1.orig
ln -s /dev/shm/coreapplication_obis1 .
– OBIS Sorting/Working directory: since temp folder is pointing the RAMDISK from above settings, use default for
sorting/working directory so it will be using RAMDISK as well
WORK_DIRECTORY_PATHS = ""; (blank means using default path)
Move tmp/log files to RAMDisks
![Page 22: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/22.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 22
Exalytics OBIS file buffer size
– Increase file buffer size to 128KB to ensure good write to disk performance of sorting and cache
Population (28.5x improvement – in 11.1.1.7 )
CACHE_FILE_BUFFER_SIZE = 128KB; (in NQConfig.INI)
I/O buffer size
![Page 23: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/23.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 23
Exalytics
– OBIPS Caches (instanceconfig.xml)
Leveraging large RAM to bump up OBI PS Caching
<Cache> <CatalogXml> <MaxAgeMinutes> : 240
<Cache> <CatalogXml> <MaxLastAccessedSeconds>: 14400
<Cache><Query><MaxEntries>: 5000
<Cache><Query><AbsoluteMaxEntries>: 20000
<Cache><Query><CruiseEntries> : 3000
<Cache><Query><ForceLRU> : true
<Cache><Accounts><CleanupFrequencyMinutes>:14400
<Cache><AccountIndex><CleanupFrequencyMinutes>:14400
<Cache><CatalogAttributes> <CleanupFrequencyMinutes>:14400
– OBIS Cache settings (NQSConfig.INI)
[CACHE]
ENABLE = YES;
MAX_CACHE_ENTRIES = 5000;
MAX_CACHE_ENTRY_SIZE = 256 MB;
MAX_ROWS_PER_CACHE_ENTRY = 100000 ; // Too many rows in cache can slow down performance
[SERVER]
INIT_BLOCK_CACHE_ENTRIES = 5000;
Caching
![Page 24: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/24.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 24
Exalytics
OBIPS Chart Threadpool size(instanceconfig.xml)
– OBIPS has a thread pool for Javahosts. Its size needs to be the total number of threads allowed in all Javahosts.
– For example, there are two Javahost instances . Each has 300 threads defined in its own configuration file, and then the Max size for
OBIPS threadpool size needs to be 600 (300 + 300).
<ThreadPoolDefaults>
<ChartThreadPool>
<MinThreads>300</MinThreads>
<MaxThreads>600</MaxThreads>
<MaxQueue>4096</MaxQueue>
</ChartThreadPool>
</ThreadPoolDefaults>
Javahost Threadpool size (config.xml)
$BI_HOME/instances/instance1/config/OracleBIJavaHostComponent/coreapplication_obijh1/config.xml
<JobManager>
<MinThreads>300</MinThreads>
<MaxThreads>300</MaxThreads>
<MaxPendingJobs>300</MaxPendingJobs>
</JobManager>
OBIS Threadpool (in NQConfig.INI)
SERVER_THREAD_RANGE = 500-1000;
DB_GATEWAY_THREAD_RANGE = 500-1000;
AGGREGATE_THREAD_POOL_SIZE = 5;
Configure Right Threadpool Size
![Page 25: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/25.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 25
Exalytics
OBIS RPD
– AdminTool: Connection Pool -> General Tab -> Maximum connections
WLS
– LDAP connection pool size
in WLS, needs to meet the concurrent login rate
Configure Right connection pool size
![Page 26: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/26.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 26
Exalytics
• WLS /analytics plug-in Max connections – bridgeconfig.properties ($BI_HOME/user_projects/domains/bifoundation_domain/config/fmwconfig/biinstances/coreapplication/bridgeconfig.properties)
oracle.bi.presentation.sawconnect.ConnectionPool.MaxConnections=512
TimesTen max connection
– Make sure you have enough Semaphore
/sbin/sysctl -w kernel.sem=2048 (to set the value of kernel.sem)
/sbin/sysctl kernel.sem (to check the value of kernel.sem)
kernel.sem = 2048 64000 256 64
– In the Timesten server side DSN entry
For example
[crm_dwh2_adm]
Driver=/home/tt/TimesTen/dev/lib/libtten.so
DatabaseCharacterSet=US7ASCII
...
Connections=300
...
• OBIS Session LIMIT (NQSConfig.INI) [SERVER]
MAX_SESSION_LIMIT = 10000 ;
Configure Max connections/Sessions
![Page 27: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/27.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 27
Exalytics
• WLS Analytics plug-in session timeout • Set the ‘Session timeout(in seconds) to control the timeout of the sessions in Analytics plug-in in WLS
• The timeout should be in sync with the session timeout in OBI PS.
OBIPS Session timeout
<ClientSessionExpireMinutes>5</ClientSessionExpireMinutes>
Check Session Timeouts
![Page 28: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/28.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 28
Exalytics
OBIPS Memory Quota Checking
– Set the following shell variable for OBIPS in opmn.xml
SIEBEL_MEMORY_THREADALLOC_QUOTA=8388608 (e.g to set thread quota to 8MB)
– UI will throw an error when a user’s request consumes more than this memory
For very large TimesTen Datastore (>256GB), The following Linux kernel and Timesten
settings need to be changed. Please refer to Exalytics Best Practices doc for details.
– kernel.shmmax/kernel.shmall/kernel.sem
– /proc/sys/vm/nr_hugepages
– /proc/meminfo: Hugepagesize
– ttendaemon.options: linuxLargePageAlignment
Exalytics Memory Tuning
![Page 29: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/29.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 29
Exalytics
• WLS BI Managed Server Heap sizes
• This needs to be modified based on the number of concurrent sessions and BIP reports.
SERVER_MEM_ARGS="-Xms256m -Xmx2048m -XX:MaxPermSize=512m“ (setDomainEnv.sh)
export SERVER_MEM_ARGS
Javahost heap sizes (opmn.xml) – For scenarios with Trellis charts , increase JVM heap size to 8GB (-xmx8192M )
<ias-component id="coreapplication_obijh1">
<data id="start-args" value="-server -Xms2048M -Xmx8192M … "
JVM heap size
![Page 30: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/30.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 30
Exalytics
Clustering Javahosts can help scalability
– Use EM to increase the number of instances of Javahost
Javahost clustering
![Page 31: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/31.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 31
Exalytics
Repository init-blocks : Called once during
server startup and refreshed after the
refresh interval as needed
Session init-blocks: called for each new
session; slow SQL can slow down login
Session init-blocks should be minimized
and lazily loaded (deferred)
– Previously it was not possible to defer row-
wise init-blocks ; restriction has been lifted in
11.1.1.6
Init-blocks should be associated with it’s
separate connection pool to reduce
contention
– Ensure enough connections are available
OBIS - Init Blocks
![Page 32: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/32.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 32
Currently BI does not support issuing SQLs with bind variables to the database
– May result in high parse time in the database specially for very complex Queries
Work around: Add inititalization code in connection setting to change session parameter for
cursor_sharing (Oracle only)
– In the Execute on connect section,
click New and add:
alter session set cursor_sharing='FORCE';
– In the Execute on disconnect section,
click New and add:
alter session set cursor_sharing='EXACT';
OBIS using Bind Variables
Exalytics
![Page 33: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/33.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 33
Exalytics OBIS Query Loglevel
– Loglevel 2 provided the logical query , physical
query and the response times
It has reasonable low overhead in low to
moderately busy system and can be kept at level 2
in production
Keep the log file in RAMDISK to reduce the IO
overhead
– Needs rpd access to change
Work-around: Have a session init-block defined
and read the value from a DB table
OBIS Log Analysis for Performance Tuning
![Page 34: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/34.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 34
Exalytics Inspecting query log
– Provides the physical datasource name [username:ADMINISTRATOR] --- Sending query to database named FSCM_OLTP (id: <<966402>>), connection pool …
– Query Status Query Status: Successful Completion
– Row Count from the database [username:Administrator] --- Execution Node: <<27109>> DbGateway Exchange, Close Row Count = 1, Row Width = 10
bytes
– Physical SQL response time [username:ADMINISTRATOR] ----- Physical query response time 5 (seconds), id SQLBypass Gateway
Mapping a slow Query in database to a logical query in BI (11.1.1.6+)
– BI physical query will add a hash as a comment - “/* 2c16620a */ select T2005.Country as c…”
– BI Query log will have the corresponding hash -------------------- SQL Request, logical request hash: fdc49e9c
Sending query to database named FSCM_OLTP (id: <<1712462>>), connection pool named Connection Pool, logical request
hash fdc49e9c, physical request hash 2c16620a: [[
OBIS Log Analysis for Performance Tuning (cont’d)
![Page 35: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/35.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 35
Exalytics
WLS Console
– # of sessions (/analytics) in WLS
EM (Enterprise manager)
– Sessions, requests/s…etc for WLS, OBIPS, OBIS & Javahost
OBIPS perfmon page
– http://<hostname>:<port>/analytics/saw.dll?perfmon
BIEE AdminTool
– Manage-Sessions: # of sessions, current request and session variables
Monitoring
![Page 36: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/36.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 36
Exalytics Monitoring – WLS Console
![Page 37: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/37.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 37
Exalytics
Monitoring – EM (Enterprise manager)
![Page 38: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/38.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 38
Exalytics Monitoring - OBIPS perfmon page /analytics/saw.dll?perfmon
![Page 39: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/39.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 39
Exalytics
Monitoring – AdminTool - Sessions
![Page 40: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/40.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 40
Exalytics
OBIPS Cursor Wait Time
– There are two parameters that control how long OBI PS should wait before replying ‘refreshing page’
to the browser. NewCursorWaitSeconds is the time to wait on the initial callback from the UI for a
query to finish. OldCursor is for subsequent call backs for the query to finish.
– It is helpful to increase the ‘NewCursorWaitSecons’ for performance load testing, so the load test
scripts won’t error out because of not seeing the expected results
<Cursors>
<NewCursorWaitSeconds>1000</NewCursorWaitSeconds>
<OldCursorWaitSeconds> 1000</OldCursorWaitSeconds>
</Cursors>
Configuration changes for Load testing scripts
![Page 41: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/41.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 41
EXALYTICS CONSIDERATIONS
![Page 42: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/42.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 42
Exalytics Version Stripe Core components – optimized, tested together
Exalytics PS1
Exalytics V1
![Page 43: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/43.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 43
Exalytics
OBIEE – Performance acceleration for Analytic Queries
Technique Expected Performance Improvement Considerations
Cached result set Consistently sub-second • For relatively static dashboard
• Pre-seeding cache, use IBOTs
• Purge cache on source data changes
• Use RAMDisk for storing the cache
Aggregate based acceleration Depends on row count reduction
• Aggregate Persistence for aggregate creation
• Summary Advisor for aggregate recommendation
• Build high-level KPI’s with summary level performance and
exception monitoring to tell a story
• Just raw data reports have no competitive advantage.
• OBIEE advantage is in multi-pass calculations, indexing,
comparisons
• Drill-down reports via Action Links to root cause analysis
Moving data into TimesTen Orders of magnitude faster depending on the queries
and the reference database
• Sizing and future data growth.
• Freeing up DB resources, in turn benefits other queries
![Page 44: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/44.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 44
Exalytics
TimesTen database is supported
data source
BI Server connects to TimesTen v11
database using TT specific ODBC
3.5
BI Server – TimesTen Connection Architecture
Server daemon
TimesTen
In-Memory
Database
BI Server
TimesTen client driver
Data Store subdaemon(s)
Admin/Utility programs
Transaction Log files
Checkpoint files
Server proxies
client/server connections
within the same host
(shared memory IPC)
![Page 45: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/45.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 45
OBIEE
Make sure the DB type is set
accurately.
– Typically more recent DB version will
support a wider range of functions.
BI Server internal execution
– Any function not supported by data
source
– Common Performance issue:
Using such function in filters
derived columns used in filters
Modeling – Function Shipping
![Page 46: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/46.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 46
OBIEE
– Use foreign key joins wherever possible
– Complex joins
Avoid
Use simple equality joins
Set right cardinality ({0..1, 1, N} to {0..1, 1, N, Unknown}) –
important for join trimming
Outer joins
– Logical level
Dimension preserving behavior
– LTS source Mapping
For all other cases
– Outer join conversion technique
– Add dimension table row for N/a or Unknown records during ETL
– Replace Null foreign keys with the N/A value either during ETL or
via IfNull expressions in the RPD
Physical and Logical Joins
![Page 47: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/47.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 47
OBIEE
defines the unique elements in each
logical level
– Portland (ME), Portland (OR) are distinct
– Basis of dimensional conformance across
sources
– Use unique, single column level keys
whenever possible
– Do not use derived columns
Consider LTS expression
Level Keys
![Page 48: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/48.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 48
OBIEE
Use Descriptor ID Columns,
– Answers Filters / Dashboard Prompts on
description
Database filters in terms of Key Columns
– Index / Partition pruning friendly SQL
SELECT
0 s_0,
"Airlines Delay"."Carrier"."Carrier" s_1,
DESCRIPTOR_IDOF( "Airlines Delay"."Carrier"."Carrier") s_2
FROM "Airlines Delay“
WHERE
(DESCRIPTOR_IDOF( "Airlines Delay"."Carrier"."Carrier")='Q5')
ORDER BY 1, 2 ASC NULLS LAST
FETCH FIRST 100001 ROWS ONLY
OBIEE - Descriptor ID columns
![Page 49: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/49.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 49
OBIEE
Runs in Online mode
Detects artifacts
– that can cause correctness issues
Uniqueness of level keys
Uniqueness of physical table keys
– that prevent BI to recommend/create
aggregates
Session Variables in LTS expression
Session Variables in LTS content filter
Generates warning
– Attributes not mapped to levels (impact on
aggregate navigation)
Model Checker
![Page 50: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/50.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 50
AGGREGATES
![Page 51: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/51.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 51
OBIEE
Ability to generate aggregate tables
based on logical star and map back
to the metadata repository
Wizard to allow the administrator to
identify measures and grains for
aggregate tables
Output is a set of scripts to create and
populate aggregate tables, as well as
automatically map sources to the rpd
Aggregate Persistence
![Page 52: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/52.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 52
OBIEE
create aggregates
"ag_4183394923“ (aggregate name)
for "Airlines"."Flight Facts“ (logical fact table)
("Actual Elapsed Time (Min)","Air Time (Min)","Arr Lag (Min)",...) (measures)
at levels
("Airlines"."00 Time"."Time Year", "Airlines"."18 Route"."18 Route Detail", "Airlines"."11 Orig Airport Map"."11 Orig Division Map") (grain – level
combination)
using connection pool
"Aggregates (TT)"."Connection Pool" (connection pool)
in "Aggregates (TT)".."AIRLINES"; (schema)
Aggregate Generation Script
![Page 53: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/53.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 53
Exalytics
Analyzes query workload over time
– Usage Tracking
– Summary Statistics table
– Cache hits are ignored.
Recommends minimal set of aggregates to
improve query response for the overall
system.
Does not actually create aggregates, but
outputs a script that is fed to the Aggregate
Persistence engine.
Summary Advisor
![Page 54: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/54.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 54
Exalytics
Analyze facts
– Eliminate the ones not suitable for
aggregation
– Few facts at a time
Analyze over specific time window
OBIEE - Summary Advisor
• Accelerating Dashboard ?
• Select logical facts accessed
![Page 55: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/55.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 55
Exalytics
Analyze facts frequently queried
Specify Aggregation Target
– Oracle
– TimesTen
OBIEE - Summary Advisor
• Workload sensitivity
• Cumulative time because of lots of small
queries or small number of time
consuming queries
• May need to modify usage tracking tables
manually to feed summary advisor right
input
![Page 56: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/56.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 56
Exalytics Summary Advisor – Stopping criteria
Default stopping criteria 0%
Longer recommendation time
Longer Aggregate build/refresh time
![Page 57: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/57.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 57
Exalytics Summary Advisor – Execute Aggregate Advisor
![Page 58: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/58.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 58
Exalytics
Update database statistics (Oracle) before running Summary Advisor
Aggregate Recommendation depends on
– Database response times
Hardware, Table sizes, Indexes, tuning etc
– Workload
Subject area, dashboard
– BI Server cache settings
Rerun Summary Advisor in case of changes in the factors
No recommendation
– Acceleration by aggregation unlikely
Reporting at the detailed grain
Reporting at the existing aggregate grain
Summary Advisor
![Page 59: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/59.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 59
OBIEE
AGO/TODATE functions will be navigated to the
detail source
– for lack of ordering information, even when the
aggregates are available
(Quarter, Sales) will hit the aggregates at
the month grain,
(Quarter, PeriodAgoSales) will not.
Specify Chronological keys for upper levels along
Time dimensions
Aggregate Navigation - Chronological keys
![Page 60: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/60.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 60
OBIEE
– E.g. Inventory measure, Account Balance etc.
– Order of Aggregation is important for correct result
– For a given query grain, matching aggregate should
honor the order.
– Consider two LTS
1. Aggregate LTS: (city, month)
2. Detailed LTS: (store, day)
Consider two queries
1. Query1: (city, year) grain
2. Query2: (state, month) grain
Only query 1 can leverage Aggregate
– “Data is dense” – permits reordering of aggregation
Review the source data before setting
Both queries will leverage aggregate.
Aggregate Navigation – Measures w/ Dimension Specific Aggregation
Country<-Region<-State<-City<-Store
Year<-Quarter<-Month<-Day
![Page 61: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/61.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 61
OBIEE
Count distinct aggregation
– BI Server uses the most detailed grain possible
– Unless you register additional LTSes explicitly
Aggregate Persistence
– Creates aggregates with the counts
– Not reusable except for Exact match with query grain
Consider expressing Distinct Count measure
as expression, referencing logical columns
– Summary Advisor can recommend suitable aggregates
Aggregate Navigation – Measures w/ Distinct Count aggregation
![Page 62: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/62.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 62
Exalytics
Aggregates on a different target ?
– VPD enabled database
– Source specific SQL [Evaluate function]
Security implemented in LTS content filter
– Consider expressing security in terms of BI logical
columns, replace with BI security filter.
– Consider Role based security
LTS expressions:
– Space - Performance tradeoff
– More columns in aggregate table – compared to base
fact [ physical tables in the database]
– Aggregate build/refresh longer, query performance
faster
Aggregates – other considerations
![Page 63: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/63.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 63
TIMESTEN
![Page 64: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/64.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 64
Exalytics
Move Select Detail Data into TimesTen
– Use ODI to replicate from warehouse tables
– Use GoldenGate to replicate for Operational reporting
– Use ttBulkCp utility to load from text file
– Data volume considerations
TimesTen in-memory database
– Space for data (PermSize) – 400 GB
– Space for query processing (TempSize) – 400 GB
Use Columnar Compression to reduce the space requirement
Trim column sizes (Currency VARCHAR 15 VARCHAR 3)
Replicate only “query” columns
Summary Advisor can be used against detailed TimesTen tables as well.
TimesTen Data mart
![Page 65: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/65.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 65
Exalytics
TimesTen implements native numeric types
– more efficient in query processing and improved space usage
BI Server automatically maps the BI NQS data types to TimesTen native data
types – INT maps to TT_INTEGER, DOUBLE maps to BINARY_DOUBLE etc.
Consider optimal data type mapping for replication to TT
– NUMBER without precision takes 22 bytes of storage
NUMBER (Max < 28) should be of type TT_TINYINT (1 Byte)
NUMBER (Max < 215) should be of type TT_SMALLINT (2 Bytes)
NUMBER (Max < 231) should be of type TT_INTEGER (4 Bytes)
TimesTen – Optimizing using native Data types
![Page 66: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/66.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 66
Exalytics
Data load:
– Disable Redo log and use fuzzy checkpoints for faster load
Maintain up to date database statistics – Exact statistics: Use the ttOptUpdateStats built-in procedure
– Statistical sampling: Use the ttOptEstimateStats built-in procedure
Indexing
– Necessary for good query performance
– BI Server currently creates single-column indexes for aggregates stored in TimesTen
– Multi-column composite indexes can provide significant improvements for complex queries
– In case of replication, use the indexes on source table as reference
– Use TimesTen Index Advisor for recommendation on indexes, for specific SQL queries/workload (nqquery.log)
TimesTen – database tuning
![Page 67: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/67.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 67
Exalytics
ttBackup creates a backup copy of a database that can be restored using the
ttRestore utility
– Full backup and incremental backup
ttBackup and ttRestore should be used to replicate across TimesTen instances
for HA/DR
TimesTen – database maintenance
![Page 68: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/68.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 68
ESSBASE
![Page 69: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/69.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 69
Exalytics
Parallel Data Load
– import database Sample.Basic using max_threads 12
Parallel Dense Restructure
– Set RESTRUCTURETHREADS in essbase.cfg for parallelism
Parallel Export
– PAREXPORT -threads 4 -in e:\data\input.txt 1 1;
– Default number of threads is specified using EXPORTTHREADS in essbase.cfg
Parallel Calculation in calculation scripts
Set CALCPARALLEL in essbase.cfg for parallelism and also set CALCTASKDIMS for the sparse
dimensions to be included in parallel tasks
ESSBASE – Block Storage - all things Parallel
![Page 70: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/70.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 70
Exalytics
Multiple user concurrent queries and calculation script execution
– Design the application to allow for large degree of parallelism in queries and calculation script
execution.
Consolidation of Applications into single Exalytics box
– Exalytics has 4 sockets, 40 cores and large amount of memory and this allows us to consolidate
number of Essbase applications in different machines into a single box leading to efficient
management.
Set Socket Affinity for Each Application
– During consolidation, explicitly set the affinity to group of applications to a single socket so as to
efficiently utilize the memory associated with that socket.
Increase the memory foot print of index and block cache to take advantage of
in-memory calculation efficiencies
ESSBASE – Block Storage - considerations
![Page 71: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/71.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 71
Exalytics
ASO scales linearly with the number of cores and memory
– Configure the ASO application to run on the entire box and assign as much memory and threads as possible
using essbase.cfg settings
Consolidation of Multiple ASO Applications into single Exalytics box
– Exalytics has 4 sockets, 40 cores and large amount of memory and this allows us to consolidate number of
Essbase ASO applications in different machines into a single box leading to efficient management.
Set Socket Affinity for Each Application
– During consolidation, explicitly set the affinity to group of applications to a single socket so as to efficiently utilize
the memory associated with that socket.
Consolidate multiple cubes into a single large cube in Exalytics box
– A large single cube is possible with the memory and CPU power of Exalytics and this avoids having to
consolidate the results from multiple cubes and re-aggregating and collating externally.
ESSBASE – Aggregate Storage - considerations
![Page 72: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/72.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 72
Complete solution for Business Analytics
Pre-packaged hardware, software,
implementation, and support
Per User Per Month (PUPM)
promotional pricing
Ask us about this special offer……..
Analytics in a Box
Contact your Sales Consultant
today to learn more.
![Page 73: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/73.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 73
Graphic Section Divider
![Page 74: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/74.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 74
![Page 75: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/75.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 75
![Page 76: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/76.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 76
Exalytics
New flag ‘HardwareAccelarator’ for Exalytics – OBIPS (instanceconfig.xml)
<OracleHardwareAcceleration>true</OracleHardwareAcceleration>
– OBIS (NQConfig.INI)
ORACLEHARDWAREACCELERATION = YES
Tip 3: HardwareAccelarator flag
![Page 77: CON9528_PDF_9528_0001](https://reader031.fdocuments.us/reader031/viewer/2022020716/55cf9a56550346d033a148fa/html5/thumbnails/77.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 77
OBIEE Architecture