Understanding Oracle RAC 12c Internals

download Understanding Oracle RAC 12c Internals

of 31

Transcript of Understanding Oracle RAC 12c Internals

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    1/31

    !"#$"%#

    %

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template 1

    UnderstandingOracle RAC 12 c InternalsMarkus Michalewicz, Mark ScardinaDirector of Product Management

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    2/31

    !"#$"%#

    '

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3

    Safe Harbor Statement

    The following is intended to outline our general product direction. It isintended for information purposes only, and may not be incorporatedinto any contract. It is not a commitment to deliver any material, code, orfunctionality, and should not be relied upon in making purchasingdecisions. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the solediscretion of Oracle.

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4

    Agenda

    ! Introduction! The New Features in Oracle Grid Infrastructure and RAC 12 c

    – Flex ASM

    – Flex Cluster

    ! Policy-Managed Databases – Improved HA Management

    – Dynamic Database Services

    !

    Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c! Summary and Q&A

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    3/31

    !"#$"%#

    #

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5

    The New Oracle RAC 12 c

    Oracle RAC 12c provides**:

    1. Better Business Continuityand High Availability (HA)

    2. Agility and Scalability

    3. Cost-effective Workload Management

    Using! Standardized and improved

    deployment and management!

    A familiar and matured HA stack

    O r a c l e

    R e a l A p

    p l i c a t i o

    n C l u s t

    e r s ( R A

    C )

    O r a c l e

    G r i d I n f r

    a s t r u c t u

    r e ( G I ) :

    A u t o m a

    t i c S t o r a

    g e M a n a

    g e m e n t

    ( A S M )

    O r a c l e

    C l u s t e r

    w a r e ( O

    C W )

    O r a c l e

    F l e x A S M

    ! " #

    **Understanding Oracle RAC Internals 11g Rel. 2 :http://www.oracle.com/technetwork/products/clustering/overview/understandingrac11grel2internals-2010098.pdf

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6

    The Setup for this SessionWhat has been installed and configured so far?

    vixen srvpool: frontofficecomet srvpool: backoffice

    dancer srvpool: backoffice dasher srvpool: frontoffice

    OS OL 6.4 OS OL 6.4

    OS OL 6.4OS OL 6.4

    Oracle GI for a cluster Oracle GI for a cluster

    Oracle GI for a clusterOracle GI for a cluster

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_2

    raccdb1_1

    [RAC]> srvctl status database -d raccdb1Instance raccdb1_1 is running on node cometInstance raccdb1_2 is running on node dancerInstance raccdb1_3 is running on node vixenInstance raccdb1_4 is running on node dasher

    raccdb1_4

    raccdb1_3

    ! 4-node cluster! OL 6.4 UEK! Oracle Grid Infrastructure 12c

    – Per node installation

    ! Oracle RAC Database Home 12c – Shared installation on ACFS

    ! One Oracle Multitenant Container Database – DB name: raccdb1

    – Deployed as a Policy-Managed Database

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    4/31

    !"#$"%#

    (

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7

    New Memory RequirementsNot an Oracle RAC internal – just something to mention

    [GRID]> srvctl status asmASM is running on comet,dasher,dancer

    [GRID]> srvctl status mgmtdbDatabase is enabledInstance -MGMTDB is running on node dasher

    [GRID]> ps -ef |grep pmongrid 3227 1 0 13:54 ? 00:00:02 asm_pmon_+ASM1grid 3868 1 0 13:55 ? 00:00:01 mdb_pmon_-MGMTDBgrid 7364 1 0 13:57 ? 00:00:01 apx_pmon_+APX1oracle 7917 1 0 13:57 ? 00:00:02 ora_pmon_raccdb1_4

    [RAC]> df -hFilesystem Size Used Avail Use% Mounted on...tmpfs 1.7G 1.3G 475M 73% /dev/shm ...

    Physical memory requirementswith Oracle RAC 12c: 4GB

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8

    Agenda

    ! Introduction! The New Features in Oracle Grid Infrastructure and RAC 12 c

    – Flex ASM

    – Flex Cluster

    ! Policy-Managed Databases – Improved HA Management

    – Dynamic Database Services

    !

    Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c! Summary and Q&A

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    5/31

    !"#$"%#

    )

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9

    Oracle ASM 12 c – OverviewOracle ASM 12 c Standard Deployment

    ASM Cluster Pool of Storage

    Disk Group BDisk Group AShared DiskGroups

    Wide File Striping

    One to OneMapping of ASMInstances toServers

    ASM Instance

    Database Instance

    ASM Disk

    RAC Cluster

    Node4Node3Node2Node1 Node5 ASM ASM ASM ASM ASM

    ASM Instance

    Database Instance

    DB A DB A DBC DBB DB B DBB

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10

    Introducing Oracle Flex ASMRemoval of One to One Mapping and HA

    ASM Cluster Pool of Storage

    Disk Group BDisk Group AShared DiskGroups

    Wide File Striping

    Databases share ASM instances

    ASM Instance

    Database Instance

    ASM Disk

    RAC Cluster

    Node5Node4Node3Node2Node1

    Node5runs as

    ASM Clientto Node4

    Node1runs as

    ASM Clientto Node2

    Node1runs as

    ASM Clientto Node4

    Node2runs as

    ASM Clientto Node3

    ASM ASM ASM

    ASM Instance

    DB A DB A DBC DBB DB B DBB

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    6/31

    !"#$"%#

    *

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11

    Supporting Pre-Oracle 12 c DatabasesPre-Oracle 12 c Databases require a local ASM instance

    ASM Cluster Pool of Storage

    Disk Group BDisk Group AShared DiskGroups

    Wide File Striping

    Databases share ASM instances

    ASM Instance

    Database Instance

    ASM Disk

    RAC Cluster

    Node5Node4Node3Node2Node1 ASM ASM ASM

    DB A DB A DBC

    ASM ASM

    11gDB

    11gDBDBB DB B DBB

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12

    Oracle Flex ASMConfiguration details

    ! Flex ASM introduces new local resources:

    – At least one ASM listener

    – One “proxy_advm” (per node)! Used for ACFS access to Flex ASM instances

    ! Connections from a database instance to an ASMinstance are based on SQLnet using listeners.

    – The listener directs the connection to the least loaded ASM instance based on the load metric it maintains.

    – The connection details are fetched from CSS global data

    – The ASM instance to which the database instanceconnects to is listed in the database alert log:

    ! NOTE: ASMB connected to ASM instance+ASM1 (Flex mode; client id 0x10004)

    – The userid and password supplied are also managedautomatically. They are supplied while establishing thesession, not while connecting.

    [GRID]> crsctl stat res -t-------------------------------------------------------------------------------Name Target State Server State details-------------------------------------------------------------------------------Local Resources-------------------------------------------------------------------------------

    ora.ASMNET1LSNR_ASM.lsnrONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLEONLINE ONLINE vixen STABLE

    ...ora.DATA.dg

    ONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLEOFFLINE OFFLINE vixen STABLE

    ...ora.INFRA.dg

    ONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLE

    OFFLINE OFFLINE vixen STABLE...

    ora.proxy_advmONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLEONLINE ONLINE vixen STABLE

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    7/31

    !"#$"%#

    +

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13

    Oracle Flex ASM Administering Flex ASM

    ! Check ASM mode using asmcmd! Check where ASM instances are running using srvctl! Check the configuration of ASM using srvctl

    ! Modify the ASM configuration using srvctl

    ! Cluster ASM listener: – Flex ASM can have its own network or overlap with the

    private interconnect network used by RAC DB instances

    – It is recommended to configure a full overlap of the privateinterconnect and the ASM network in current use cases:

    [GRID]> asmcmdASMCMD> showclustermodeASM cluster : Flex mode enabled

    [GRID]> srvctl status asm ASM is running on comet,dasher,dancer

    [GRID]> srvctl config asm -detailASM home: /u01/app/12.1.0/gridPassword file: +INFRA/orapwASMASM listener: LISTENERASM is enabled.ASM instance count: 3Cluster ASM listener: ASMNET1LSNR_ASM,ASMNET2LSNR_ASM

    [GRID]> srvctl modify asm -count all

    [GRID]> srvctl status asm ASM is running on comet,dasher,dancer

    [GRID]> srvctl start asm

    [GRID]> srvctl status asm ASM is running on comet,vixen,dasher,dancer

    Test scenarios attached

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14

    Oracle Flex ASMSome test scenarios !

    ! A failure of an ASM instance is without effect for thedatabases using ASM disk groups (here: -MGMTDB)

    – If an ASM instance is running on all nodes (count all)the ASM instance is restarted upon failure

    – If an ASM instance is running on a subset of nodesthe ASM instance might be restarted on another node

    – The client using the ASM instance may change

    ! A failure of an ASM instance is without effect forthe APX (ADVM proxy) running on the same node

    ! A failure of the APX will lead to a restart of the databaseinstance (on the node) that uses an ACFS file system.

    ! More information: – http://docs.oracle.com/cd/E16655_01/server.121/e17612/

    asminst.htm#OSTMG95329

    [GRID]> ps -ef |grep pmongrid 3840 1 0 09:11 ? 00:00:01 mdb_pmon_-MGMTDBgrid 4093 1 0 11:35 ? 00:00:00 asm_pmon_+ASM1grid 4548 1 0 11:36 ? 00:00:00 apx_pmon_+APX1oracle 5092 1 0 11:36 ? 00:00:00 ora_pmon_raccdb1_4

    [GRID]> kill -9 4093[GRID]> ps -ef |grep pmongrid 3840 1 0 09:11 ? 00:00:01 mdb_pmon_-MGMTDBgrid 4548 1 0 11:36 ? 00:00:00 apx_pmon_+APX1oracle 5092 1 0 11:36 ? 00:00:00 ora_pmon_raccdb1_4

    [GRID]> ps -ef |grep pmongrid 3840 1 0 09:11 ? 00:00:01 mdb_pmon_-MGMTDBgrid 4548 1 0 11:36 ? 00:00:00 apx_pmon_+APX1oracle 5092 1 0 11:36 ? 00:00:00 ora_pmon_raccdb1_4grid 5748 1 0 11:39 ? 00:00:00 asm_pmon_+ASM1grid 5793 19013 0 11:39 pts/1 00:00:00 grep pmon

    [GRID]> asmcmd lsct dataDB_Name Status Software_Version Compatible_version Instance_Name Disk_Group+APX CONNECTED 12.1.0.1.0 12.0.0.0.0 +APX1 DATA

    [GRID]> ssh vixen /u01/app/12.1.0/grid/bin/asmcmd lsct dataDB_Name Status Software_Version Compatible_version Instance_Name Disk_Group+APX CONNECTED 12.1.0.1.0 12.0.0.0.0 +APX2 DATA+APX CONNECTED 12.1.0.1.0 12.0.0.0.0 +APX3 DATA

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    8/31

    !"#$"%#

    ,

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15

    New in Oracle Grid Infrastructure 12cOracle Flex Cluster for better resource management

    D B A D B A

    D B A D B B

    D B B

    Hub Nodes

    Leaf Nodes

    ! Oracle Flex Cluster: – The new standard for flexible

    resource management in the cluster! Use it as demand requires it! Requires Oracle Flex ASM to be enabled! Enables policy-managed clusters.

    – Efficient High Availability andresource management across tiers

    ! Lightweight cluster stack on leaf nodes

    – Benefit from virtualization andphysical deployments in one cluster

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16

    Oracle Flex ClusterSome configuration details

    ! Oracle Flex Cluster

    – Will be the basis for more uses casesin future releases (for resource mgmt)

    – Can consist of only HUB-nodes

    – If a Leaf-node has access to shared storage,it can perform a role change to a HUB-node

    – A Standard Cluster can be converted to aFlex Cluster at any time after the installation

    – Once a cluster is configured as a Flex Clusterthis configuration is irreversible without a re-configuration of the cluster.

    ! More Information: – http://docs.oracle.com/cd/E16655_01/rac.121/e17886/

    bigcluster.htm#CWADD92560

    [GRID]> crsctl get cluster mode configCluster is configured as "standard"

    [GRID]> crsctl get cluster mode statusCluster is running in "standard" mode

    [GRID]> olsnodes -s -a -tdasher Active Hub Unpinnedvixen Active Hub Unpinneddancer Active Hub Unpinnedcomet Active Hub Unpinned

    • Not a Flex Cluster yet.• An all-Hub Flex Cluster is

    basically a Standard Cluster

    Example attached

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    9/31

    !"#$"%#

    !

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17

    Oracle Flex ClusterConverting to Flex Cluster

    ! More Information: – http://docs.oracle.com/cd/E16655_01/rac.121/e17886/

    bigcluster.htm#CHDJGBII

    ! Flex Cluster implementations require GNS! Converting a Flex Cluster requires a cluster restart

    – Use “crsctl stop crs” followed by “crsctl start crs” as root

    – Hence the difference between “status” and “config”

    – Nodes can be restarted in a rolling fashion:

    [GRID]> srvctl config gnsPRKF-1110 : Neither GNS server nor GNS client is configured onthis cluster

    [GRID]> more /etc/hosts |grep gns10.1.1.99 mygns mygns.localdomain

    #Change to root

    [GRID]> suPassword:[GRID]> srvctl add gns -vip mygns[GRID]> srvctl start gns

    [GRID]> crsctl set cluster mode flexCluster mode set to "flex »

    [GRID]> crsctl get cluster mode statusCluster is running in "standard" mode

    [GRID]> crsctl get cluster mode configCluster is configured as "flex"

    [GRID]> date ; hostname ;crsctl get cluster mode statusSat Sep 14 14:43:45 PDT 2013dasherCluster is running in "flex" mode

    [GRID]> date ; hostname ;crsctl get cluster mode statusSat Sep 14 14:43:37 PDT 2013dancerCluster is running in "standard" mode

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18

    Oracle Flex ClusterStandard to Flex Cluster conversion – what has changed?

    vixen srvpool: frontofficecomet srvpool: backoffice

    dance r srvpool: backoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

    [GRID]> crsctl get cluster mode statusCluster is running in "flex" mode

    [GRID]> olsnodes -a -s -tdasher Active Hub Unpinnedvixen Active Hub Unpinneddancer Active Hub Unpinnedcomet Active Hub Unpinned

    [RAC]> srvctl status database -d raccdb1Instance raccdb1_1 is running on node cometInstance raccdb1_2 is running on node dancerInstance raccdb1_3 is running on node vixenInstance raccdb1_4 is running on node dasher

    ! Nothing at first glance ! – An all-Hub Flex Cluster behaves

    like a Standard Cluster

    – When converting from Standard Clusterall nodes are converted into HUB-nodes

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    10/31

    !"#$"%#

    %$

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19

    Oracle Flex ClusterFlexibility and High Availability as demand requires it

    ! Flex Cluster utilizes two types of cluster nodes: – Hub nodes

    ! Traditional nodes, tightly coupled via network andstorage, used for running IO intensive workload

    – Leaf nodes! A new type of node that runs a lighter weight stack! Maintain their own failure and heartbeat settings! Do not require direct shared storage connectivity! Used for processing oriented (application) workload

    Hub Nodes

    Leaf Nodes

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20

    Oracle Flex ClusterChanging node roles – server vixen becomes a Leaf node

    ! More Information: – http://docs.oracle.com/cd/E16655_01/rac.121/e17886/

    bigcluster.htm#BABHCEAE

    [GRID]> hostnamevixen

    [GRID]> srvctl config vip -node vixenVIP exists: network number 1, hosting node vixenVIP Name: vixen-vipVIP IPv4 Address: 10.1.1.33VIP IPv6 Address:

    [GRID]> crsctl get node role configNode 'vixen' configured role is 'hub'

    #Change to root …[GRID]> suPassword:

    [GRID]> crsctl set node role leafCRS-4408: Node 'vixen' configured role successfully changed; restartOracle High Availability Services for new role to take effect.

    [GRID]> crsctl stop crs[GRID]> crsctl start crs

    [GRID]> crsctl get node role configNode 'vixen' configured role is 'leaf'

    [GRID]> srvctl config vip -node vixenVIP exists: network number 1, hosting node vixenVIP Name: vixen-vipVIP IPv4 Address: 10.1.1.33VIP IPv6 Address:

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    11/31

    !"#$"%#

    %%

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21

    Oracle Flex ClusterNode role change to Leaf – what has changed?

    vixen srvpool: FREEcomet srvpool: backoffice

    dance r srvpool: backoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | LEAFOracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_4 raccdb1_2

    raccdb1_1[RAC]> srvctl status database -d raccdb1Instance raccdb1_1 is running on node cometInstance raccdb1_2 is running on node dancerInstance raccdb1_4 is running on node dasher

    [RAC]> srvctl status server -servers vixen -detailServer active pools: FreeServer name: vixenServer state: ONLINEServer state details:

    ! Server vixen has converted to a Leaf node! Leaf nodes do not host DB instances in 12.1! Server vixen moved to Server Pool “FREE” as

    Server Pool “frontoffice” is a database pool – Note: all nodes in this example have access

    to shared storage, which means only the noderole changes – storage access is preserved.

    Example attached

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22

    Oracle Flex ClusterRevert role change – server vixen becomes a HUB again

    comet srvpool: backoffice

    dance r srvpool: backoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RAC

    raccdb1

    raccdb1_4 raccdb1_2

    raccdb1_1

    vixen srvpool: frontofficeOracle GI | HUB

    Oracle RAC

    raccdb1_3

    #As root[GRID]> crsctl set node role hubCRS-4408: Node 'vixen' configured role successfully changed;restart Oracle High Availability Services for new role to takeeffect.

    [GRID]> crsctl stop crs[GRID]> crsctl start crs

    [RAC]> srvctl status database -d raccdb1Instance raccdb1_1 is running on node cometInstance raccdb1_2 is running on node dancerInstance raccdb1_3 is running on node vixenInstance raccdb1_4 is running on node dasher

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    12/31

    !"#$"%#

    %'

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23

    Agenda

    ! Introduction! The New Features in Oracle Grid Infrastructure and RAC 12 c

    – Flex ASM

    – Flex Cluster

    ! Policy-Managed Databases – Improved HA Management

    – Dynamic Database Services

    ! Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c! Summary and Q&A

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24

    Policy-Managed Databases with Oracle RAC 12 c Adding another dimension – Database Services

    vixen srvpool: frontoffice

    dance r srvpool: backoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    [GRID]> srvctl status serverpoolServer pool name: frontoffice

    Active Servers count: 2Server pool name: backoffice

    Active Servers count: 2

    [RAC]> srvctl status service -d raccdb1Service crmsvc is running on nodes: dasher,vixenService hrsvc is running on nodes: comet,dancer

    ! Database Services – Ensure that workload is hosted in the respective

    server pool with the cardinality defined as part ofthe policy definition.

    comet srvpool: backoffice Oracle GI | HUB

    Oracle RAC

    raccdb1_1

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    13/31

    !"#$"%#

    %#

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25

    Policy-Managed Databases with Oracle RAC 12 cImproved HA Management - Startup

    raccdb1

    [GRID]> srvctl config serverpoolServer pool name: frontofficeImportance: 10, Min: 2, Max 2Server pool name: backofficeImportance: 5, Min: 1, Max 1 Server pool name: FreeImportance: 0, Min: 0, Max -1

    ! Defining a Service Startup Order – Ensure services are started in specified groups

    and specified order.

    – Leverages Serverpool Min and Importance properties

    dasher srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_4

    1 st

    vixen srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_3

    2nd

    comet srvpool: backoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_1

    4 th

    dance r srvpool: backoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    3 rd

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26

    dancer srvpool: backoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    dancer srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    Policy-Managed Databases with Oracle RAC 12 cImproved HA Management – New Failover Strategy

    raccdb1

    [GRID]> srvctl config serverpoolServer pool name: frontofficeImportance: 10, Min: 2, Max 2Server pool name: backofficeImportance: 5, Min: 1, Max 1

    [GRID]> srvctl status serverpoolServer pool name: frontoffice

    Active Servers count: 2Server pool name: backoffice Active Servers count: 1

    dasher srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_4

    vixen srvpool: frontofficeOracle GI | HUB

    Oracle RAC

    raccdb1_3

    comet srvpool: backoffice Oracle GI | HUB

    Oracle RAC

    raccdb1_1

    ! Servers “Move” to Replace Failed node – Protects against cascade failures due to load

    – Ensures workload isolation between pools

    – Less important workloads shutdown transactionally

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    14/31

    !"#$"%#

    %(

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27

    dancer srvpool: backoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    dancer srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    Policy-Managed Databases with Oracle RAC 12 cImproved HA Management – Manage Last Service Standing

    raccdb1

    [RAC]> srvctl config serverpoolServer pool name: frontofficeImportance: 10, Min: 2, Max 2Server pool name: backofficeImportance: 5, Min: 1, Max 1

    [RAC]> srvctl status service –dbService crmsrv is running on nodes comet,vixenService hrsvc is not running

    dasher srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_4

    vixen srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_3

    comet srvpool: backoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_1

    ! Business Critical Services survive multiple failures – Most important pool always gets the servers

    – Services preserved across multiple failures

    – Less important workloads shutdown transactionally

    comet srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_1

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28

    Policy-Managed Databases with Oracle RAC 12 cImproved HA Management – Dynamic Provisioning

    vixen srvpool: frontoffice

    dance r srvpool: backoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    [GRID]> srvctl modify serverpool–serverpool backoffice –max 1

    [RAC]> srvctl config serverpoolServer pool name: frontofficeImportance: 10, Min: 2, Max 4Server pool name: backofficeImportance: 5, Min: 1, Max 1

    [RAC]> srvctl status service –dbService crmsrv is running on nodes dasher,dancer,vixenService hrsvc is not running

    comet srvpool: backoffice Oracle GI | HUB

    Oracle RAC

    raccdb1_1

    ! Add Servers Just-In Time to meet demand – Server Pools sized via Min and Max properties

    – Dynamically controlled by QoS Management

    – Planned control via Clusterware or QoSM Policies

    dancer srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    15/31

    !"#$"%#

    %)

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29

    Agenda

    ! Introduction! The New Features in Oracle Grid Infrastructure and RAC 12 c

    – Flex ASM

    – Flex Cluster

    ! Policy-Managed Databases – Improved HA Management

    – Dynamic Database Services

    ! Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c! Summary and Q&A

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30

    Policy-Management with Oracle RAC 12 cSupport for Multiple Policies tracking business objectives

    ! More Information: – http://docs.oracle.com/cd/E16655_01/rac.121/e17886/pbmgmt.htm

    ! New in Oracle Grid Infrastructure 12 c – Server Categories

    ! Server Categories use server attributes to allowfor an active use of differently sized servers

    – Policy Sets! Policy Sets allow for dynamic adjustment

    to demand changes in an atomic transaction

    Server Categories• NAME• ACL• EXPRESSION• …

    Server Attributes• NAME• MEMORY_SIZE • CPU_COUNT• CPU_CLOCK_RATE • CPU_HYPERTHREADING • CPU_EQUIVALENCY• …

    [GRID]> crsctl modify policyset –attr"LAST_ACTIVATED_POLICY=NightTime“CRS-2673: Attempting to stop'ora.raccdb1.crmsvc.svc' on 'comet'CRS-2673: Attempting to stop'ora.raccdb1.crmsvc.svc' on 'dancer'CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on'comet' succeededCRS-2673: Attempting to start'ora.raccdb1.backup.svc' on 'comet'CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on'dancer' succeededCRS-2672: Attempting to start'ora.raccdb1.hrsvc.svc' on 'dancer‘…

    QoS Management

    OR Oracle Clusterware - CRSCTL

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    16/31

    !"#$"%#

    %*

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31

    Policy-Management with Oracle RAC 12 cSetting up policy sets – Provision Server Pools and creating a PolicySet

    ! Add another server pool “backup”

    ! Set up policy set with 3 server pools & 3 policies as follows:

    – DayTime:! frontoffice uses three servers (MIN_SIZE=3)

    ! backoffice uses one server (MIN_SIZE=1)

    ! backup does not run during daytime (MIN_SIZE=0)

    – NightTime:! frontoffice uses one server (MIN_SIZE=1)

    ! backoffice uses two servers (MIN_SIZE=2)! backup uses only one server (MIN_SIZE=1)

    – Weekend:

    ! frontoffice uses one server (MIN_SIZE=1)! backoffice uses one server (MIN_SIZE=1)

    ! backup uses two servers (MIN_SIZE=2)

    [GRID]> srvctl add serverpool–serverpool backup –min 0 –max 2 –importance 20

    [GRID] srvctl status serverpoolServer pool name: frontoffice

    Active Servers count: 3Server pool name: backoffice

    Active Servers count: 1 Server pool name: backup

    Active Servers count: 0

    [GRID]> crsctl modify policyset-file /u01/app/…/policyset.txt

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32

    Policy-Management with Oracle RAC 12 cUsing Policy Sets – part 1: Activate a policy

    ! Activate the policy for the weekend

    [GRID]> crsctl modify policyset-attr "LAST_ACTIVATED_POLICY= Weekend "

    [GRID]> srvctl config serverpoolServer pool name: FreeImportance: 0, Min: 0, Max: -1Category:Candidate server names:Server pool name: GenericImportance: 0, Min: 0, Max: -1Category:Candidate server names:Server pool name: backofficeImportance: 0, Min: 1, Max: 1Category:Candidate server names:Server pool name: backup

    Importance: 1, Min: 2, Max: 2Category:Candidate server names:Server pool name: frontofficeImportance: 0, Min: 1, Max: 1Category:Candidate server names:

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    17/31

    !"#$"%#

    %+

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33

    Policy-Management with Oracle RAC 12 cUsing Policy Sets – part 2: check the result

    vixen srvpool: frontofficecomet srvpool: backup

    dance r srvpool: backup dasher srvpool: backoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

    [RAC]> srvctl status database -d raccdb1Instance raccdb1_1 is running on node cometInstance raccdb1_2 is running on node dancerInstance raccdb1_3 is running on node vixenInstance raccdb1_4 is running on node dasher

    [RAC]> srvctl status service -d raccdb1

    Service backup is running on nodes: comet,dancerService crmsvc is running on nodes: vixenService hrsvc is running on nodes: dasher

    ! With an activated weekend policy

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34

    Policy-Management with Oracle RAC 12 c Adding Server Categories to the picture – part 1

    ! Assume you have 2 servers that have better IO – Use these servers for backups whenever possible

    ! Here comet and dancer have better IO by definition

    ! What you need to do: – Set up a server category that identifies the servers

    – Add the use of the server category to the server pool! Define the server pools that utilize the category

    and during which policy activation it shall be used.

    ! You need to restart the cluster stack onthe servers that you modify in this fashion

    [GRID]> suPassword:[GRID]> crsctl set server label IOplus...# On dancer[GRID]> crsctl set server label Ioplus

    [GRID]> crsctl get server labelCRS-4972: Current SERVER_LABEL parameter value is Ioplus

    [GRID]> crsctl status server comet dancer –f Comet Dancer

    NAME=cometMEMORY_SIZE=3338CPU_COUNT=1CPU_CLOCK_RATE=2132CPU_HYPERTHREADING=0CPU_EQUIVALENCY=1000DEPLOYMENT=otherCONFIGURED_CSS_ROLE=hubRESOURCE_USE_ENABLED=1SERVER_LABEL=IOplus...

    NAME=dancerMEMORY_SIZE=3338CPU_COUNT=1CPU_CLOCK_RATE=2247CPU_HYPERTHREADING=0CPU_EQUIVALENCY=1000DEPLOYMENT=otherCONFIGURED_CSS_ROLE=hubRESOURCE_USE_ENABLED=1SERVER_LABEL=IOplus...

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    18/31

    !"#$"%#

    %,

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35

    Policy-Management with Oracle RAC 12 c Adding Server Categories to the picture – part 2

    ! What you need to do: – Set up a server category that identifies the servers -

    done

    – Add the use of the server category to the server pool! Define the server pools that utilize the category

    and during which policy activation it shall be used

    ! Note : Add the server category usage to theserver pool(s) that require it & specify thepolicies that should be altered accordingly

    – In this example the backup server pool is the one

    that will benefit the most and the policies will onlystart this server pool as required.

    [GRID]> crsctl add category moreIO–attr "EXPRESSION='SERVER_LABEL co IOplus'"

    [GRID]> crsctl status category moreIONAME=moreIOACL=owner:grid:rwx,pgrp:oinstall:rwx,other::r--ACTIVE_CSS_ROLE=hubEXPRESSION=(SERVER_LABEL co IOplus)

    [GRID]> crsctl modify serverpool ora.backup-attr "SERVER_CATEGORY=moreIO" -all_policies

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36

    Policy-Management with Oracle RAC 12 c Adding Server Categories to the picture – comparison

    Before After

    [GRID]> crsctl status policy DayTimePOLICY

    NAME=DayTimeDESCRIPTION=SERVERPOOL

    NAME=Free...

    SERVERPOOLNAME=ora.backofficeIMPORTANCE=0MAX_SIZE=1MIN_SIZE=1SERVER_CATEGORY=SERVER_NAMES=

    SERVERPOOLNAME=ora.backupIMPORTANCE=0MAX_SIZE=0MIN_SIZE=0SERVER_CATEGORY=SERVER_NAMES=

    SERVERPOOLNAME=ora.frontofficeIMPORTANCE=0MAX_SIZE=3MIN_SIZE=3SERVER_CATEGORY=SERVER_NAMES=

    [GRID]> crsctl status policy DayTimePOLICY

    NAME=DayTimeDESCRIPTION=SERVERPOOL

    NAME=Free...

    SERVERPOOLNAME=ora.backofficeIMPORTANCE=0MAX_SIZE=1MIN_SIZE=1SERVER_CATEGORY=SERVER_NAMES=

    SERVERPOOLNAME=ora.backupIMPORTANCE=0MAX_SIZE=0MIN_SIZE=0SERVER_CATEGORY=moreIO

    SERVER_NAMES=SERVERPOOLNAME=ora.frontofficeIMPORTANCE=0MAX_SIZE=3MIN_SIZE=3SERVER_CATEGORY=SERVER_NAMES=

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    19/31

    !"#$"%#

    %!

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37

    Policy-Management with Oracle RAC 12 cUsing Policy Sets and the effect of Server Categories

    vixen srvpool: frontofficecomet srvpool: backup

    dance r srvpool: backup dasher srvpool: backoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

    vixen srvpool: backofficecomet srvpool: frontoffice

    dancer srvpool: frontoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38

    [GRID]> crsctl modify policyset –attr"LAST_ACTIVATED_POLICY=DayTime“

    [GRID] srvctl status serverpoolServer pool name: frontoffice

    Active Servers count: 3 Server pool name: backoffice

    Active Servers count: 1Server pool name: backup

    Active Servers count: 0

    [RAC]> srvctl status service -d raccdb1Service backup is not running.Service crmsvc is running on nodes: dasher,vixen,cometService hrsvc is running on nodes: dancer

    Policy-Management with Oracle RAC 12 cUsing Policy Sets means changing policies on a push of a button

    raccdb1

    vixen srvpool: frontofficeOracle GI | HUB

    Oracle RAC

    raccdb1_3

    dasher srvpool: frontoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_4

    comet srvpool: backup Oracle GI | HUB

    Oracle RAC

    raccdb1_1

    dance r srvpool: backup

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    dancer srvpool: backoffice

    Oracle GI | HUB

    Oracle RAC

    raccdb1_2

    comet srvpool: frontofficeOracle GI | HUB

    Oracle RAC

    raccdb1_1

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    20/31

    !"#$"%#

    '$

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39

    Policy-Management with Oracle RAC 12 cUsing Policy Sets – changing policies on the push of a button is fast !

    vixen srvpool: frontofficecomet srvpool: backup

    dance r srvpool: backup dasher srvpool: backoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

    vixen srvpool: backofficecomet srvpool: frontoffice

    dancer srvpool: frontoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

    [GRID]> date; crsctl modify policyset -attr"LAST_ACTIVATED_POLICY=DayTime"; date

    Mon Sep 16 19:26:42 PDT 2013CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'dancer'CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'comet'CRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'dancer' succeededCRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'comet' succeededCRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'dancer'CRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'comet'CRS-2676: Start of ' ora.raccdb1.crmsvc.svc' on 'dancer' succeededCRS-2676: Start of ' ora.raccdb1.crmsvc.svc' on 'comet' succeededMon Sep 16 19:26:43 PDT 2013

    Time to execute: 1 second!

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40

    Policy-Management with Oracle RAC 12 cWhat-If evaluation of policy changes

    [RAC]> srvctl status service -d raccdb1Service backup is not running.Service crmsvc is running on nodes: dancer,vixen,dasherService hrsvc is running on nodes: comet

    [GRID]> crsctl eval activate policy Weekend Stage Group 1:-------------------------------------------------------------------------------Stage Number Required Action-------------------------------------------------------------------------------

    1 Y Server 'comet' will be moved from pools[ora.frontoffice] to pools [ora.backup]

    Y Server 'dancer' will be moved from pools[ora.frontoffice] to pools [ora.backup]

    Y Resource 'ora.raccdb1.crmsvc.svc' (1/1) will bein state [OFFLINE]

    Y Resource 'ora.raccdb1.crmsvc.svc' (2/1) will bein state [OFFLINE]

    2 Y Resource 'ora.raccdb1.backup.svc' (1/1) will bein state [ONLINE|INTERMEDIATE] on server[comet]

    Y Resource 'ora.raccdb1.backup.svc' (2/1) will bein state [ONLINE|INTERMEDIATE] on server[dancer]

    vixen srvpool: frontofficecomet srvpool: frontoffice

    dance r srvpool: backoffice dasher srvpool: frontoffice

    Oracle GI | HUB Oracle GI | HUB

    Oracle GI | HUB Oracle GI | HUB

    Oracle RAC Oracle RAC

    Oracle RACOracle RAC

    raccdb1

    raccdb1_3

    raccdb1_4 raccdb1_2

    raccdb1_1

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    21/31

    !"#$"%#

    '%

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41

    What-If with Oracle RAC 12 cWhat-If evaluation of policy changes – in various levels

    [RAC]> srvctl status service -d raccdb1Service backup is not running.Service crmsvc is running on nodes: dancer,vixen,dasherService hrsvc is running on nodes: comet

    [GRID]> crsctl eval activate policy Weekend -admin -l 'serverpools'

    NAME = FreeACTIVE_SERVERS =

    NAME = GenericACTIVE_SERVERS =

    NAME = ora.backofficeACTIVE_SERVERS = vixen

    NAME = ora.backupACTIVE_SERVERS = comet dancer

    NAME = ora.frontofficeACTIVE_SERVERS = dasher

    [RAC]> srvctl status service -d raccdb1Service backup is not running.Service crmsvc is running on nodes: dancer,vixen,dasherService hrsvc is running on nodes: comet

    [GRID]> crsctl eval activate policy Weekend -admin -l 'resources'

    -------------------------------------------------------------------------------- Name Target State Server Effect

    --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.mgmtdb

    1 ONLINE ONLINE dasherora.raccdb1.backup.svc

    1 ONLINE ONLINE comet Started2 ONLINE ONLINE dancer Started

    ora.raccdb1.crmsvc.svc1 ONLINE OFFLINE Stopped2 ONLINE OFFLINE Stopped3 ONLINE ONLINE dasher

    ora.raccdb1.db1 ONLINE ONLINE comet2 ONLINE ONLINE dancer3 ONLINE ONLINE vixen4 ONLINE ONLINE dasher

    ora.raccdb1.hrsvc.svc1 ONLINE ONLINE vixen2 ONLINE OFFLINE

    --------------------------------------------------------------------------------

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42

    Policy-Management with Oracle RAC 12 cInformation on each step on the way

    Before After

    [GRID]> crsctl modify policyset -attr"LAST_ACTIVATED_POLICY=NightTime“

    CRS-2673: Attempting to stop 'ora.raccdb1.crmsvc.svc' on 'dancer'CRS-2673: Attempting to stop 'ora.raccdb1.crmsvc.svc' on 'comet'CRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on 'dancer' succeededCRS-2677: Stop of 'ora.raccdb1.crmsvc.svc' on 'comet' succeededCRS-2672: Attempting to start 'ora.raccdb1.backup.svc' on 'dancer'CRS-2672: Attempting to start 'ora.raccdb1.backup.svc' on 'comet'CRS-2676: Start of 'ora.raccdb1.backup.svc' on 'dancer' succeededCRS-2676: Start of 'ora.raccdb1.backup.svc' on 'comet' succeeded

    [GRID]> crsctl modify policyset -attr"LAST_ACTIVATED_POLICY=DayTime"

    CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'dancer'CRS-2673: Attempting to stop 'ora.raccdb1.backup.svc' on 'comet'CRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'dancer' succeededCRS-2677: Stop of 'ora.raccdb1.backup.svc' on 'comet' succeededCRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'dancer'CRS-2672: Attempting to start 'ora.raccdb1.crmsvc.svc' on 'comet'CRS-2676: Start of 'ora.raccdb1.crmsvc.svc' on 'dancer' succeededCRS-2676: Start of 'ora.raccdb1.crmsvc.svc' on 'comet' succeeded

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    22/31

    !"#$"%#

    ''

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43

    Agenda

    ! Introduction! The New Features in Oracle Grid Infrastructure and RAC 12 c

    – Flex ASM

    – Flex Cluster

    ! Policy-Managed Databases – Improved HA Management

    – Dynamic Database Services

    ! Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c! Summary and Q&A

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44

    Multiple Subnet Support in Oracle RAC 12 cPreparing for multiple subnet usage – part 1

    ! Multiple subnet support was introducedwith Oracle RAC 11g Rel. 2

    – Using multiple subnets per cluster can come inhandy for various reasons – backups for example.

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network ora.net1.network

    [GRID]> crsctl stat res ora.net1.network -t------------------------------------------------------------------------Name Target State Server State details------------------------------------------------------------------------Local Resources------------------------------------------------------------------------ora.net1.network

    ONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLEONLINE ONLINE vixen STABLE

    ------------------------------------------------------------------------

    [GRID]> oifcfg iflisteth0 10.1.1.0eth1 192.168.7.0eth1 169.254.128.0eth2 172.149.2.0eth2 169.254.0.0

    eth3 10.2.2.0[GRID]> oifcfg getifeth0 10.1.1.0 global publiceth1 192.168.7.0 global cluster_interconnect,asmeth2 172.149.2.0 global cluster_interconnect,asm

    One more physical interfacethat can be used for public or

    private communication.

    Private communication alreadyuses multiple subnets.

    Example attached

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    23/31

    !"#$"%#

    '#

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.45

    Multiple Subnet Support in Oracle RAC 12 cPreparing for multiple subnet usage – part 2

    ! Multiple subnet support can beactivated in two alternative ways:

    1. Defining interface name and subnet identifier

    2. Defining only a subnet identifier (interface: “*”)

    ! Using approach one allows for more consistentmanagement and will be used in this example.

    [GRID]> oifcfg setif -global eth3/10.2.2.0:public

    [GRID]> oifcfg getifeth0 10.1.1.0 global publiceth1 192.168.7.0 global cluster_interconnect,asmeth2 172.149.2.0 global cluster_interconnect,asmeth3 10.2.2.0 global public

    [GRID]> oifcfg setif -global "*"/10.2.2.0:public

    [GRID]> oifcfg getifeth0 10.1.1.0 global publiceth1 192.168.7.0 global cluster_interconnect,asm

    eth2 172.149.2.0 global cluster_interconnect,asm* 10.2.2.0 global publicPRIF-29: Warning: wildcard in network parameters cancause mismatch among GPnP profile, OCR, and system

    Alternative configuration:

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46

    Multiple Subnet Support in Oracle RAC 12 c Adding second network resource to support multiple public subnets

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    ora.net2.network:10.2.2.0 ora.net2.network:10.2.2.0

    [GRID]> suPassword:

    [GRID]> srvctl add network -netnum 2-subnet 10.2.2.0/255.255.255.0

    [GRID]> exit

    [GRID]> crsctl stat res ora.net2.network -t------------------------------------------------------------------------Name Target State Server State details------------------------------------------------------------------------Local Resources------------------------------------------------------------------------ora.net2.network

    OFFLINE OFFLINE comet STABLEOFFLINE OFFLINE dancer STABLEOFFLINE OFFLINE dasher STABLE

    OFFLINE OFFLINE vixen STABLE------------------------------------------------------------------------

    ! The preparation: – Prepare the physical connectivity – done

    – Prepare the stack as shown

    ! Adding the network is one command! VIPs need to be added separately.

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    24/31

    !"#$"%#

    '(

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47

    Multiple Subnet Support in Oracle RAC 12 c Adding VIPs on second network to support multiple public subnets

    ! Adding node VIPs to second subnet: [GRID]> more /etc/hosts

    ...

    10.2.2.30 dasher-vip2 dasher-vip2.localdomain

    10.2.2.31 dancer-vip2 dancer-vip2.localdomain

    10.2.2.32 comet-vip2 comet-vip2.localdomain

    10.2.2.33 vixen-vip2 vixen-vip2.localdomain

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    ora.net2.network ora.net2.network

    [GRID]> srvctl config vip -node cometVIP exists: network number 1, hosting node cometVIP Name: comet-vipVIP IPv4 Address: 10.1.1.32VIP IPv6 Address:

    [GRID]> srvctl config vip -node vixenVIP exists: network number 1, hosting node vixenVIP Name: vixen-vipVIP IPv4 Address: 10.1.1.33VIP IPv6 Address:

    [GRID]> suPassword:[GRID]> srvctl add vip -node comet -k 2-A comet-vip2/255.255.255.0

    [GRID]> srvctl add vip -node vixen -k 2-A vixen-vip2/255.255.255.0

    [GRID]> srvctl add vip -node dancer -k 2-A dancer-vip2/255.255.255.0

    [GRID]> srvctl add vip -node dasher -k 2-A dasher-vip2/255.255.255.0

    vixen-vip / 10.1.1.33 comet-vip / 10.1.1.32

    vixen-vip2 / 10.2.2.33 comet-vip2 / 10.2.2.32

    Complete Example attached

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48

    Multiple Subnet Support in Oracle RAC 12 cStart using the second subnet VIPs

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    ora.net2.network ora.net2.network

    #Example only – need to check for all nodes [GRID]> srvctl config vip -node vixenVIP exists: network number 1, hosting node vixenVIP Name: vixen-vipVIP IPv4 Address: 10.1.1.33VIP IPv6 Address:VIP exists: network number 2, hosting node vixenVIP Name: vixen-vip2VIP IPv4 Address: 10.2.2.33VIP IPv6 Address:

    [GRID]> srvctl status vip -node vixenVIP vixen-vip is enabledVIP vixen-vip is running on node: vixenVIP vixen-vip2 is enabledVIP vixen-vip2 is not running

    [GRID]> srvctl start vip -node vixenPRKO-2420 : VIP is already started on node(s): vixen

    [GRID]> srvctl start vip -vip vixen-vip2[GRID]> srvctl start vip -vip comet-vip2[GRID]> srvctl start vip -vip dancer-vip2[GRID]> srvctl start vip -vip dasher-vip2

    vixen-vip / 10.1.1.33 comet-vip / 10.1.1.32

    vixen-vip2 / 10.2.2.33 comet-vip2 / 10.2.2.32

    ! Once VIPs are added verify the setup

    ! Start the VIPs in order to use them! Note: you need to specify the VIP name for the

    VIPs on the second subnet to be started, as thedefault assumes the “default node VIP”.

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    25/31

    !"#$"%#

    ')

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49

    Listeners in Oracle RAC 12 cThe default listener is available by default

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    [GRID]> srvctl status listener Listener LISTENER is enabledListener LISTENER is running on node(s):comet,vixen,dasher,dancer

    [GRID]> srvctl config listenerName: LISTENERNetwork: 1, Owner: gridHome: End points: TCP:1521

    LISTENER: 10.1.1.13and 10.1.1.33

    LISTENER: 10.1.1.10and 10.1.1.30

    ! Various listeners are used in Oracle RAC 12 c! These are all listeners – used in a certain way!

    ! The simplest one is the default listener! A listener depends on a network! A listener will listen on the node and VIP IP

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.50

    Listeners in Oracle RAC 12 cThe default listener FAQ

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    [GRID]> srvctl modify listener -hModifies the configuration for the listener.

    Usage: srvctl modify listener [-listener ][-oraclehome ] [-endpoints "[TCP:] [, ...][/IPC:][/NMP:][/TCPS:] [/SDP:]"] [-user ] [-netnum ]

    #Example – adding another listener on another port on net 1 [GRID]> srvctl add listener -listener LSNR2NET1 -netnum 1[GRID]> srvctl config listener -listener LSNR2NET1Name: LSNR2NET1Network: 1, Owner: gridHome: End points: TCP:1522

    [GRID]> srvctl status listener -listener LSNR2NET1Listener LSNR2NET1 is enabledListener LSNR2NET1 is not running

    [GRID]> srvctl start listener -listener LSNR2NET1[GRID]> srvctl status listener -listener LSNR2NET1Listener LSNR2NET1 is enabledListener LSNR2NET1 is running on node(s):comet,vixen,dasher,dancer

    ! Can you define another port? - YES! Can you define another owner? – YES! Can you define another home? – YES , if needed.! Can I add another listener on another port? - YES

    LISTENER LISTENER

    LSNR2NET1 LSNR2NET1

    Example attached

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    26/31

    !"#$"%#

    '*

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.51

    Listeners in Oracle RAC 12 cThe default listener FAQ

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    [GRID]> srvctl modify listener -listener LSNR2NET1-endpoints "TCP:1528«

    [GRID]> lsnrctl status LSNR2NET1

    LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 16-SEP-2013 19:34:13

    Copyright (c) 1991, 2013, Oracle. All rights reserved.

    Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LSNR2NET1)))STATUS of the LISTENER------------------------Alias LSNR2NET1Version TNSLSNR for Linux: Version 12.1.0.1.0 - ProductionStart Date 16-SEP-2013 16:35:34

    Uptime 0 days 2 hr. 58 min. 39 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/12.1.0/grid/network/admin/liste ner.oraListener Log File /u01/app/oracle/diag/tnslsnr/dasher/lsnr 2net1/alert/log.xmlListening Endpoints Summary...

    (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LSNR2NET1)))(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.1.1.10)( PORT=1528 )))(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.1.1.30)( PORT=1528 )))

    Services Summary...

    ! Can you define another port? - YES

    ! New in Oracle Grid Infrastructure 12 c:! Online Modification of Resources

    LISTENER LISTENER

    LSNR2NET1:1522 LSNR2NET1:1522 LSNR2NET1: 1528 LSNR2NET1: 1528

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.52

    Listeners in Oracle RAC 12 c Adding a node listener to the second subnet

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    [GRID]> srvctl add listener -listener LSNR1NET2-netnum 2 -endpoints 1522

    [GRID]> srvctl config listener -listener LSNR1NET2Name: LSNR1NET2

    Network: 2, Owner: gridHome: End points: TCP:1522

    [GRID]> srvctl status listener -listener LSNR1NET2Listener LSNR1NET2 is enabledListener LSNR1NET2 is not running

    [GRID]> srvctl start listener -listener LSNR1NET2[GRID]> srvctl status listener -listener LSNR1NET2Listener LSNR1NET2 is enabledListener LSNR1NET2 is running on node(s):comet,vixen,dasher,dancer

    ! Can I add another listener on another subnet? ! Absolutely!

    ora.net2.network ora.net2.network LSNR1NET2:1522 LSNR1NET2:1522

    Listener summary attached

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    27/31

    !"#$"%#

    '+

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.53

    Listeners in Oracle RAC 12 cOther listeners used as part of the stack

    [GRID]> crsctl stat res –t--------------------------------------------------------------------------------Name Target State Server State details--------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------

    ora.ASMNET1LSNR_ASM.lsnrONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLEONLINE ONLINE vixen STABLE

    ora.ASMNET2LSNR_ASM.lsnrONLINE ONLINE comet STABLEONLINE ONLINE dancer STABLEONLINE ONLINE dasher STABLEONLINE ONLINE vixen STABLE

    ...--------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------

    ora.LISTENER_SCAN1.lsnr1 ONLINE ONLINE dasher STABLE

    ora.MGMTLSNR1 ONLINE ONLINE dasher 169.254.73.11 192.16

    8.7.10 172.149.2.10 ,STABLE

    ! Oracle RAC 12 c introduces more listener (types):! ora.LISTENER.lsnr

    ! Default listener of type “ora.listener.type”! ora.ASMNET n LSNR_ASM.lsnr

    ! An Oracle ASM listener used by Flex ASM of type “ora.asm_listener.type”

    ! ora.mgmtlsnr.type! Grid Infrastructure Management Repository

    (GIMR) DB listener of type “ora.mgmtlsnr.type”! ora.LISTENER_SCAN n. lsnr”

    ! A SCAN listener of type “ora.scan_listener.type”

    local resource (runs on every node)

    cluster resource (runs on any node)

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.54

    Listeners in Oracle RAC 12 cValid Node Checking for Registration

    ! New in the Oracle RAC 12 c listeners:! For Oracle RAC: IPv6 support! Valid Node Checking for Registration

    ! http://docs.oracle.com/cd/E16655_01/network.121/e17610/listenercfg.htm#NETAG1588

    ! The listener is now protected to both sides:! It can be restricted to accept client connections

    ! http://docs.oracle.com/cd/E16655_01/network.121/e17607/guidelines.htm#DBSEG504

    ! It can be restricted to prevent ay arbitrary DB(instance) from registering to it, if not allowedby default. (Valid Node Checking for Registration)

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    LISTENER LISTENER

    raccdb1_3raccdb1_1

    Valid NodeChecking forRegistration

    SCAN

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    28/31

    !"#$"%#

    ',

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.55

    SCAN in Oracle RAC 12 cThe listener usage that combines it all

    [GRID]> srvctl config networkNetwork 1 existsSubnet IPv4: 10.1.1.0/255.255.255.0/eth0, staticSubnet IPv6:

    [GRID]> srvctl config scan_listenerSCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521Registration invited nodes:Registration invited subnets :

    ...

    [GRID]> srvctl status scan_listener -netnum 2SCAN Listener LISTENER_SCAN1_NET2 is enabledSCAN listener LISTENER_SCAN1_NET2 is running on node vixen

    ! SCAN was introduced with Oracle RAC 11g Rel. 2 – http://www.oracle.com/technetwork/products/

    clustering/overview/scan-129069.pdf

    ! New in SCAN with Oracle RAC 12 c : – Multiple Subnet Support

    ! IPv6 support for client connectivity (public network)

    – Valid Node Checking for Registration

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    ora.net2.network ora.net2.network MySCAN_3 / 10.1.1.52

    MySCAN2_3 / 10.2.2.52 MySCAN2_2 / 10.2.2.51

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.56

    SCAN in Oracle RAC 12 c Adding a second SCAN listener to the system

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    ora.net2.network ora.net2.network MySCAN / 10.1.1.50

    ! Adding a second SCAN listener to the system – There is only one SCAN listener per subnet

    – In this example, each SCAN has only 1 IP address

    ! Three steps are required to add another scan:1. Preparation of the host / network

    2. Adding another SCAN (VIPs mainly)

    3. Adding another SCAN listener

    #Step 1: Preparation[GRID]> more /etc/hosts ...10.1.1.50 myscan myscan.localdomain10.2.2.50 myscan2 myscan2.localdomain

    [GRID]> srvctl config networkNetwork 1 existsSubnet IPv4: 10.1.1.0/255.255.255.0/eth0, staticSubnet IPv6:

    Network 2 existsSubnet IPv4: 10.2.2.0/255.255.255.0/, staticSubnet IPv6:

    The second (non-default)network to be used for the

    second scan (netnumber 2)

    A second IP for SCAN2

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    29/31

    !"#$"%#

    '!

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.57

    SCAN in Oracle RAC 12 cStarting the SCAN listener on the second network

    #Step 2: Adding another SCAN[GRID]> suPassword:

    [GRID]> srvctl add scan -scanname myscan2 -netnum 2[GRID]> exit

    [GRID]> srvctl add scan_listener -netnum 2 -endpoints 1530

    [GRID]> srvctl status scan -netnum 2SCAN VIP scan1_net2 is enabledSCAN VIP scan1_net2 is not running

    [GRID]> srvctl status scan_listener -netnum 2SCAN Listener LISTENER_SCAN1_NET2 is enabledSCAN listener LISTENER_SCAN1_NET2 is not running

    [GRID]> srvctl start scan_listener -netnum 2[GRID]> srvctl status scan_listener -netnum 2SCAN Listener LISTENER_SCAN1_NET2 is enabledSCAN listener LISTENER_SCAN1_NET2 is running on node vixen

    vixencomet

    Oracle GI | HUB Oracle GI | HUB

    Oracle RACOracle RAC

    ora.net1.network:10.1.1.0 ora.net1.network:10.1.1.0

    ora.net2.network ora.net2.network MySCAN / 10.1.1.50

    ! Then start the SCAN on the second network

    LISTENER_SCAN1_NET2

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.58

    SCAN in Oracle RAC 12 cStart using SCAN with the Oracle Database

    [RAC]> sqlplus sys/ @MySCAN:1521/raccdb1 as sysdba...

    SQL> show parameters listener

    NAME TYPE VALUE------------------------------------ ----------- ------------------------------listener_networks stringlocal_listener string (ADDRESS=(PROTOCOL=TCP)(HOST=

    10.1.1.31)(PORT=1521)), (ADDRESS=(PROTOCOL=TCP)(HOST=10.1.1.31)(PORT=1528))

    remote_listener string myscan:1521

    [RAC]> srvctl stop database -d raccdb1[RAC]> srvctl start database -d raccdb1[RAC]> sqlplus sys/oracle@MySCAN:1521/raccdb1 as sysdba ...SQL> show parameters listener

    NAME TYPE VALUE------------------------------------ ----------- ------------------------------

    listener_networks string (( NAME=net2)(LOCAL_LISTENER=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.2.2.32)(PORT=1522))))), ((NAME=net2)(REMOTE_LISTENER=myscan2:1530))

    local_listener string (ADDRESS=(PROTOCOL=TCP)(HOST=10.1.1.32)(PORT=1521)), (ADDRESS=(PROTOCOL=TCP)(HOST=10.1.1.32)(PORT=1528))

    remote_listener string myscan:1521

    ! When using more than one SCAN on the system,the LISTENER_NETWORKS parameter needs tobe set for databases.

    ! New in Oracle RAC12 c : – The LISTENER_NETWORKS parameter

    is managed by the Oracle (Database) Agentas part of the Oracle Clusterware management.

    – To initiate the new configurationa database restart is required.

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    30/31

    !"#$"%#

    #$

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.59

    SCAN in Oracle RAC 12 cThe SCAN listener FAQ

    ! Can you define another port? - YES! Can I define multiple ports? – YES**

    ! Can you define another owner? – NO! Can you define another home? – NO

    ! Can I add another listener on another port? – YES ! See MOS Note 220970.1 – The RAC FAQ:

    ! How to use SCAN and nodelisteners with different ports?

    • ** For the purpose of allowing applications tomaintain a distinguishable connect string on host /port level using one SCAN as the host or toenable port-based firewall configurations, SCANcan be set up to support more than one port.

    • It needs to be noticed that different ports are onlyused for an initial connect.

    • Once a connection using SCAN on any SCANport has been established, port assignment is notmaintained .

    • This means that all SCAN ports are treated thesame, so that a connection that comes in on anySCAN port can be directed to any registered locallistener on any port.

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved.60

    Agenda

    ! Introduction! The New Features in Oracle Grid Infrastructure and RAC 12 c

    – Flex ASM

    – Flex Cluster

    ! Policy-Managed Databases – Improved HA Management

    – Dynamic Database Services

    !

    Multiple Subnet Support, Listeners & SCAN in Oracle RAC 12c! Summary and Q&A

  • 8/19/2019 Understanding Oracle RAC 12c Internals

    31/31

    !"#$"%#

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template 61

    Graphic Section Divider

    Copyright © 2013, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 of the corporate presentation template 62