12c Multi-Tenancy and Exadata IORM Presented by: Fahd Mirza Chughtai

© 2012 Pythian

About Fahd Mirza (mirza@pythian.com)Oracle DBA since 2001Oracle ACEOracle Press Co-Author of 12c BookFrequent Oracle Conference SpeakerWrite Weekly Log Buffer for PythianAustralian Certified TrainerLives in Australia@fahdmirza (Twitter)Working at PythianInterviewed by Oracle MagazineOracle Blogger (fahdmirza.blogspot.com) since 2005

© 2012 Pythian

Recognized Leader:• Global industry-leader in remote database administration services and

consulting for Oracle, Oracle Applications, MySQL and SQL Server• Work with over 200 multinational companies such as Forbes.com, Fox Sports,

Nordion and Western Union to help manage their complex IT deployments

Expertise:• One of the world’s largest concentrations of dedicated, full-time DBA expertise.

Employ 7 Oracle ACEs/ACE Directors• Hold 7 Specializations under Oracle Platinum Partner program, including Oracle

Exadata, Oracle GoldenGate & Oracle RAC

Global Reach & Scalability:• 24/7/365 global remote support for DBA and consulting, systems administration,

special projects or emergency response

Why Pythian

© 2012 Pythian

• Cloud, Exadata, Consolidation• Resource Management in Container Database• Database Resource Manager• I/O Resource Manager• Methods of IORM• Monitoring IORM• Conclusion

Agenda

© 2012 Pythian5

Cloud Computing and Exadata• Cloud clouds the complexity of IT.• Exadata clouds the complexity of Servers, storage, operating system, and RDBMS by integrating and optimizing them.• Consolidation is the cornerstone of Cloud computing.• Consolidation in cloud is standardization and limiting the number of distinct environment. • The business objectives achieved are greater agility, less risk, and lower costs.

© 2012 Pythian6

Cloud Computing and Exadata (cont'd)• Exadata is an engineered cloud in a box.• Standardized components of Exadata facilitate flexibility, consistency, security, and scalability.• Granularity increases because engineered systems like Exadata / Exalogic support a specific functions.• Exadata offers specialization because it performs only well defined duties and its really good at them.• Capacity Planning and performance tuning are more predictable. • Rapid system stand-up and production implementation are common because its already built.• Single point of contact vendor.

© 2012 Pythian7

Oracle ExadataTwo Prime Reasons to Have Exadata:• Consolidation• Performance

Caution: Exadata is not a place to dump and forget data!

© 2012 Pythian8

Oracle Exadata – A Powerful Elixir, but…• Not an Infinite Black box• Common Factors behind hype• Temptation to abuse Oracle Exadata• Requires proactive planning • Requires on-going resource management• You must harness the power

© 2012 Pythian9

Resource Management Solution• Complexity of balancing resources among databases

• Resource Allocation Priorities• Enablers are:

• DBRM for Database Node• IORM for Cell Node

© 2012 Pythian10

Who is the Unsung Hero?Hints:• Present for a long time, but used not that often• Implementation seems daunting• People can’t believe it really works• Mostly Ignored by DBAs, Devs, Mkt and Bloggers• But things are improving now steadily

© 2012 Pythian11

Yes, It’s DBRM• DBRM requires KISS• Horses and Donkeys are same without it• Should CEO’s query to determine his next bonus run with equal priority as

my query to determine my next loan installment?

• Without resource management, ad-hoc queries can make critical applications suffer

© 2012 Pythian12

DBRM adds Value!

DBRM is for automating resource management • DBAs address resource management within the database itself• Even during heavy load periods, critical applications get resources

© 2012 Pythian13

DBRM in Multi-Tenant Database• DBRM is multi-tenant aware. • Enables to rank the PDBs according to the business importance• DBRM decides on the CDB level and on the PDB level.• On CDB level, DBRM manages the resources between multiple PDBs.• On PDB level, DBRM manages resources within that PDB among consumers.

© 2012 Pythian14

DBRM in Multi-Tenant Database (cont'd)• DBRM limits the resource usage of different sessions connected to a single PDB• DBRM limits the CPU usage of a particular PDB• DBRM limits the number of parallel execution servers that a particular PDB can use• DBRM enables you to monitor the resource usage of PDBs

© 2012 Pythian15

CDB Plan• A CDB plan allocates resources to the PDBs.• In a CDB plan, each directive references to a single PDB.• CDB plan directive controls allocation of CPU and parallel execution servers• CDB plan deals in share values and utilization limits to for precise control over resource allocation for PDBs.• When there is no CDB resource plan, the CDB uses the DEFAULT_CDB_PLAN that is supplied with Oracle Database.

© 2012 Pythian16

Shares and Utilization Limits (caps)• Shares are like counters. • More the shares a PDB has, the more resources it would enjoy.• The utilization limit for a PDB limits resource allocation to the PDB.• By default, share value for each PDB is 1, and utilization limit is 100.

© 2012 Pythian17

Shares• For example, we have 2 databases: prod and dev. • Prod database has 2 shares, whereas Dev database has 1 share.• It means that prod enjoys double the resources of Dev database.• In terms of CPU, Prod is guaranteed to have double the CPU of Dev.• In terms of Parallel Execution, Queued parallel queries from the Prod database will be selected to be executed two times more often than the Dev.

© 2012 Pythian18

Utilization Limits (caps)• Utilization limit restrains the system resource usage of a specific PDB.• Utilization limits for CPU and parallel execution servers can be specified.• The utilization_limit parameter limits CPU usage for a PDB use. The value ranges from 0 to 100%.• Parallel_server_limit is used to limit parallel servers.• If a PDB crosses the threshold of its utilization limit, the DBRM throttles it back in terms of CPU and queued statements.• For example, if utilization limit of Prod database is 100, then it has no utilization limit.• If utilization limit of Dev database is 50, then it can use 50 % of system resources.

© 2012 Pythian19

PDB Plan• A PDB resource plan determines how the resources allocated to a specific PDB are allocated to consumer groups within that PDB.• A PDB resource plan is similar to a resource plan for a non-CDB.• A PDB resource plan allocates resource among the consumer groups within a PDB.• A PDB resource plan cannot have subplans.• A PDB resource plan can have a maximum of eight consumer groups.

© 2012 Pythian20

Create CDB Plan for PDBsexec DBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA();BEGIN DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN( plan => 'cdb_plan', comment => 'CDB plan for CDB');END;

BEGIN DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN_DIRECTIVE( plan => 'cdb_plan', pluggable_database => 'Prod', shares => 2, utilization_limit => 100, parallel_server_limit => 100);END;

© 2012 Pythian21

Create CDB Plan for PDBs (cont'd)

BEGIN DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN_DIRECTIVE( plan=> 'cdb_plan', pluggable_database => 'Dev', shares=> 1, utilization_limit => 50, parallel_server_limit => 50);END;

exec DBMS_RESOURCE_MANAGER.VALIDATE_PENDING_AREA();exec DBMS_RESOURCE_MANAGER.SUBMIT_PENDING_AREA();

© 2012 Pythian22

CDB Plan for PDBs• Use the DBA_CDB_RSRC_PLANS view to display all of the CDB resource plans defined in the CDB.• Use the DBA_CDB_RSRC_PLAN_DIRECTIVES view to display all of the directives.• The DEFAULT_CDB_PLAN is a default CDB plan that is supplied with Oracle Database.• To enable the plan, use:

• ALTER SYSTEM SET RESOURCE_MANAGER_PLAN = 'cdb_plan';

• Or set parameter RESOURCE_MANAGER_PLAN = 'cdb_plan' in initialization file.

© 2012 Pythian23

Components of DBRM in PDBConsumer Group• Sessions with similar resource requirements• Resources are allocated to consumer groups

Plan Directive• Resource allocations are defined in plan directives• Consumer group is assigned to plan directive

Plan• Plan directives are grouped to create a resource plan• Define the holistic allocation of resources to all sessions within the

database

© 2012 Pythian24

DBRM: Consumer GroupsDBRM is in-play in the database automatically• SYS and SYSTEM use consumer group SYS_GROUP• Other sessions by default use OTHERS_GROUP

Manage resources on the fly by dynamically changing consumer groups of sessions

A session may belong to many consumer groups• A consumer group can have many sessions assigned to it

All sessions in a consumer group share the resources of that group

© 2012 Pythian25

DBRM: Consumer Group Mapping RulesWhen a resource plan is activated, DBRM examines key attributes of all sessions in the database and assigns them to consumer groups

• Assigned by mapping rules that you defineExample: a set of user accounts can be mapped to a consumer group based on user name or the login machine:

BEGINDBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(DBMS_RESOURCE_MANAGER.ORACLE_USER, 'SCOTT', 'FINANCE');DBMS_RESOURCE_MANAGER.SET_CONSUMER_GROUP_MAPPING(DBMS_RESOURCE_MANAGER.CLIENT_PROGRAM, ‘rep.exe', ‘RPTAPP');END;

© 2012 Pythian26

DBRM: Mapping Rules Session AttributesTwo Types:• login attributes: used to determine which consumer group the session

should initially be assigned to • Common Login Attributes are :

ORACLE_USERSERVICE_NAMECLIENT_MACHINE

• Runtime Attribute: the runtime attribute commonly used is MODULE_NAME

DBMS_SESSION.SWITCH_CURRENT_CONSUMER_GROUP• explicitly switch a user to another consumer group

© 2012 Pythian27

DBRM: Plan DirectivesDBRM allocates database resources to consumer groups through plan directives:• Consists of one consumer group and one or more management attributes

and other parameters.• In directive, these parameters are listed in a key=value format• One-to-one relationship between a plan directive and a consumer group• Contains parameters for limiting CPU, logical I/O, physical I/O, degree of

parallelism(PARALLEL_DEGREE_LIMIT_Pn), concurrently active sessions(ACTIVE_SESS_POOL_Pn), undo pool (undo generated in KB i.e. undo_pool), idle time for session before its terminated etc.

© 2012 Pythian28

DBRM: Sample Plan DirectiveCPU can be controlled through Mgmt_Pn attributes among consumer groups on basis of levels and percentages.

DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE(PLAN => 'my_plan',GROUP_OR_SUBPLAN => 'REPORTING',COMMENT => 'DW Sessions',MGMT_P1 => 70,MAX_UTILIZATION_LIMIT => 90);

© 2012 Pythian29

MGMT_Pn for CPU on levels

© 2012 Pythian30

DBRM I/O Limits• DBRM enables you to control I/O for runaway queries.• Both physical I/O and logical I/O can be limited.• If SWITCH_FOR_CALL directive is TRUE, then I/O requests are accumulated from start of call, otherwise from start of session.• For example:• SWITCH_IO_REQS: Limits number of physical I/O requests a session can

execute.• SWITCH_IO_LOGICAL: Limits number of logical I/O requests a session can

execute.

© 2012 Pythian31

DBRM I/O Limits (cont'd)• After I/O Threshold is reached, a session may:• Switch to another consumer group• switch_group => ‘<Group Name>'

• Or its SQL may be cancelled• switch_group => ‘CANCEL_SQL'

• Or the session it self may be killed• switch_group => 'KILL_SESSION’

• Or only session info is recorded in SQL Monitoring• switch_group => ‘LOG_ONLY’

• Column RM_LAST_ACTION in V$SQL_MONITOR specified recent action taken on this SQL by DBRM like KILL_SESSION, CANCEL_SQL, LOG_ONLY

© 2012 Pythian32

DBRM I/O Limits (cont'd)

BEGIN DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE ( PLAN => 'LIOPLAN', GROUP_OR_SUBPLAN => 'MYGROUP', COMMENT => 'Limit the Logical IO', MGMT_P1 => 75, SWITCH_GROUP => 'OTHER_GROUP', SWITCH_IO_LOGICAL => 100, SWITCH_FOR_CALL => TRUE);END;/

© 2012 Pythian33

DBRM: Resource Plan• A collection of plan directives that determine how database resources are allocated• A database may have many resource plans but only one will be active• Switching between resource plans provides required allocations for particular times of the day, week, or month

© 2012 Pythian34

DBRM: Pending Area• Pending area is a staging area for resource plan

BEGINDBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA(); <create, modify, delete your resource plan>DBMS_RESOURCE_MANAGER.VALIDATE_PENDING_AREA(); DBMS_RESOURCE_MANAGER.SUBMIT_PENDING_AREA(); END;

© 2012 Pythian35

DBRM Limitations

Limited to managing resources within the databaseNo awareness of storage tier by itself

© 2012 Pythian36

Enter IORM

© 2012 Pythian37

IORM• Oracle strengthens presence at storage tier

• Directly Manage I/O at Cell level

• I/O Request by databases through iDB to cellsrv

• Using iDB, the database packs additional attributes in every I/O call to the storage cells

• IORM manages I/O allowances configured in cell

• IORM automatically manages IO from background processes like LGWR, control file, instance recovery etc.

© 2012 Pythian38

IORM (cont'd)Intelligent I/O requests from database to cellsrv

• contains three important attributes: database name, consumer group, category

• Priorities can be assigned at cell level with help of these attributes

• Common categories across databases share same I/O priority

© 2012 Pythian39

IORM at WorkOrganize IO Requests in queues

Queues = db, category, group etc

Only manages IO when needed

IO queues serviced in FIFO when cell is idle

In off-peak cell gives maximum throughput

In rush-hour cell IO requests are scheduled and prioritize by IORM to disks.

All IO processing goes through cellsrv

© 2012 Pythian40

IORM Architecture

idb

IORM

CELL Node

Cellsrv IO Queues

w.r.t DB, CG,

bkgrnd procs etc

Cell Disk IO

Queues

DB Node

© 2012 Pythian41

IORM Workload OptimizationIORM distinguishes between small and large I/O requests

• Requests less than 128K in size are small (SM) requests

• Requests 128K in size or greater are large (LG) requests

• SM Requests are OLTP and LG Requests are DW for IORM

© 2012 Pythian42

DB Node Knows About Request Types

SQL> SELECT function_name FUNCTION, filetype_name , small_read_megabytes SM_READ_MB, LARGE_READ_MEGABYTES LG_READ_MBFROM v$iostat_function_detail;

FUNCTION FILETYPE_NAME SM_READ_MB LG_READ_MB-------- ------------- ---------- ----------RMAN Control File 2219 52974RMAN Data File 128 16467663RMAN Archive Log 2 5956878RMAN Other 0 0DBWR Control File 7 0DBWR Data File 1 0LGWR Control File 36 0LGWR Log File 0 0

© 2012 Pythian43

IORM Optimization ObjectivesOptimization objective attribute determines optimization mode IORM will use when managing cell disk I/O queuesLow_Latency

• Favors OLTP, DW Suffers. • Use this setting for critical OLTP workloads that require extremely good disk latency.

This setting provides the lowest possible latency by significantly limiting disk utilization.

High_Throughput• Favors DW, OLTP Suffers. Use this setting for DW workload that require high

throughput.

Balanced• For Mixed Workloads. This setting balances low disk latency and high throughput.

Auto• Leave it to IORM. IORM continuously and dynamically determines the

optimization objective, based on the workloads observed

Basic• Limit the maximum SMALL I/O(< 128K) Latency

© 2012 Pythian44

Setting ObjectiveNo objective set initially ( Before Storage Server software 11.2.3.2)

Use Cellcli to set IORM Objective:CellCLI> alter iormplan objective = low_latencyIORMPLAN successfully altered

CellCLI> list iormplan attributes objectivelow_latency

CellCLI> list iormplan detail; name: mycell catPlan: dbPlan: objective: off status: inactive

© 2012 Pythian45

Setting Objective (cont'd)From Exadata Storage Software 11.2.3.2 and above, the default value is ‘basic’.‘Basic’ objective only User-defined resource manager plans are not enforced in this mode.  To enable IORM for user-defined resource manager plans, the objective must be set to "auto" (or any setting other than "basic").

© 2012 Pythian46

IORM : Methods of IO Management

I/O Management among Databases• Interdatabase IORM: based on database name

I/O Management by Workload Type• IORM Categories: based on categories

I/O Management For a Database• Intradatabase IORM: based on consumer groups

© 2012 Pythian47

IORM Interdatabase I/O Resource Management• Manages I/O priority among multiple databases by database name

• Allocates I/O resources across multiple databases by means of an IORM plan configured on each storage cell

• The IORM plan defines allocation priorities using I/O directives.• Each storage cell manages its own I/O queues based on the local IORM plan

defined in the cell

© 2012 Pythian48

Configuring Inter-database IORM

CellCLI> ALTER IORMPLAN -dbPlan=( -(name=db1, level=1, allocation=60), -(name=db2, level=2, allocation=80), -(name=db3, level=3, allocation=100))

• The LEVEL attribute specifies the priority a database should be given relative to other databases in the plan

• The ALLOCATION attribute determines the percentage of I/O a database will be given out of the total I/O available on its level

© 2012 Pythian49

IORM Category I/O Resource ManagementCategories provide I/O resource management among multiple databases• It’s like sharing priorities among databases• Categories represent collections of consumer groups across all databases.• We create categories in database, assign those categories to consumer

groups

© 2012 Pythian50

Configuring Category I/O Resource ManagementCreate category:• dbms_resource_manager.create_category(category =>

'APPS_CATEGORY‘,comment => 'Category for Interactive Applications');

Assign Consumer Groups to Categories:• dbms_resource_manager.update_consumer_group(consumer_group =>

'APPS‘,new_category => 'APPS_CATEGORY');

To check your resource-group-to-category mappings, query the DBA_RSRC_CONSUMER_GROUPS view

Then create the Category IORM plan:• CellCLI> alter iormplan catplan=((name=APPS_CATEGORY, level=1,

allocation=70), (name=BATCH_CATEGORY, level=1, allocation=30), (name=OTHER, level=2, allocation=100))

Now multiple databases with the above same categories will have same resource allocation at specific level.

© 2012 Pythian51

IORM Intradatabase I/O Resource Management• Configure DBRM, and it will be pushed to cell• Intradatabase I/O resource management is configured in each database using DBRM’s management attributes, MGMT_P1..8 etc• The database transmits a description of the plan, including MGMT_Pn directives, to all cells in the storage grid• IORM will generate an Intradatabase Plan and begin managing I/O priorities according to plan directives in database

© 2012 Pythian52

Monitoring IORMMonitoring and measuring essential for tweaking resource managementI/O performance metrics are collected and maintained for IORM in the storage cellEvery 60 seconds, cellsrv collects key performance metrics from the I/O queues and stores them in the METRICCURRENT object

© 2012 Pythian53

Monitoring IORM (cont'd)Metric history is maintained in METRICHISTORY object

• METRICHISTORY retention is controlled by METRICHISTORYDAYS attribute of cell

CellCLI> list cell attributes name, metricHistoryDaysmycell 7

CellCLI> alter cell metricHistoryDays='30'Cell mycell successfully altered

© 2012 Pythian54

Monitoring IORM (cont'd)In METRICCURRENT object, the NAME of the metric is:

CT_IO_RQ_R_SM_SEC

METRIC OBJECT TYPE

CL-- Cell�CD-- Cell Disk�GD-- Grid DiskFC-- Flash Cache�DB-- IORM Database�CG-- IORM Consumer GroupCT-- IORM Category

IO METRIC DETAILS

IO_RQ-- # requestsIO_BY-- # MBIO_TM-- IO latencyIO_WT-- Wait Time�

R--read operation�W--write operationLG-- Large IO�SM-- Small IO

Rate

_SEC: The metric is normalized to per second

_RQ: The metric is normalized to per (IO) request

© 2012 Pythian55

Monitoring IORM (cont'd)CellCLI> describe metriccurrentnamealertStatecollectionTimemetricObjectNamemetricTypemetricValueobjectType

CellCLI> LIST METRICCURRENT –WHERE name = 'CT_IO_RQ_LG' –AND objecttype = 'IORM_CATEGORY' –Name objectType metricObjectName metricValue-------------- -------------- ----------------- -------------------CT_IO_RQ_LG IORM_CATEGORY OLTP_CATEGORY 56 IO requestsCT_IO_RQ_LG IORM_CATEGORY REPORTING_CATEGORY 21,111 IO requestsCT_IO_RQ_LG IORM_CATEGORY OTHER 1452 IO requests

© 2012 Pythian56

Simple Ways to Monitor MetricsEM 12c UI displays the metrics in nice graphsFor CLI, Oracle provides metric_iorm.pl script• Doc ID 1337265.1 Script is valid for all version of storage cells.Script helps in determining:• IORM effect on the database and consumer groups.• Disk I/O latency• Flash Cache hit ratio (For OLTP workloads, It should be high)• Script can be run for specific time periods, relative time periods• Script can obtain data from single cell or from multiple cells.

© 2012 Pythian57

Simple Ways to Monitor Metrics (cont'd)

./metric_iorm.pl "where ageInMinutes > 60 and ageInMinutes < 120"

Database: exaDBUtilization: Small=1% Large=53%Flash Cache: IOPS=1830Disk Throughput: MBPS=902Small I/O's: IOPS=176 Avg qtime=0.0msLarge I/O's: IOPS=885 Avg qtime=0.0ms

Consumer Group: exaDW_GROUPUtilization: Small=0% Large=48%Flash Cache: IOPS=248Disk Throughput: MBPS=814Small I/O's: IOPS=11.3 Avg qtime=0.0msLarge I/O's: IOPS=784 Avg qtime=0.0ms

CELL METRICS SUMMARYCell Total Utilization: Small=1% Large=53%Cell Total Flash Cache: IOPS=1830Cell Total Disk Throughput: MBPS=860.971

Cell Avg small read latency: 49.28 msCell Avg small write latency: 37.91 msCell Avg large read latency: 59.46 msCell Avg large write latency: 9.86 ms

-- Utilization show how busy disks are--IOPS are I/O rate

per second-- disk latency is in

msec--avg qtime is time waited by IORM to

schedule I/O

© 2012 Pythian

Conclusion

Maturing IT industry has become very cost conscious• Resource management is a key DBA skillOracle offers tools that leverage resources at the database and cell level• Risk: they can be confusing and get out of controlMillion dollar tip: keep it very simple and straight forward• Implement a simple, single-level Resource Plan, and grow from

there (most systems don't need to though)Don't be afraid of it. Use it!

© 2012 Pythian

Thank you and Q&A

http://www.pythian.com/news/

http://www.facebook.com/pages/The-Pythian-Group/163902527671

@pythian

http://www.linkedin.com/company/pythian

1-877-PYTHIAN

sales@pythian.com

To contact us…

To follow us…

@pythianjobs

WE ARE HIRING !!!