ES-331 Sun Enterprise Cluster Administration Veritas Volume Manager

Sun Educational Services

Sun Enterprise Cluster Administration – Veritas Volume Manager October 1999

Sun Enterprise ClusterAdministration – Veritas Volume

Manager

ES-331


Sun Enterprise Cluster Administration – Veritas Volume ManagerCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999 Revision A

Copyright © 1999 Sun Microsystems, Inc., 901 San Antonio Road, Palo Alto, California 94303, U.S.A. All rights reserved.

This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of thisproduct or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and othercountries, exclusively licensed through X/Open Company, Ltd.

Sun, Sun Microsystems, the Sun Logo, Sun Enterprise, Sun StorEdge Volume Manager, Solstice DiskSuite, Solaris Operating Environment, Sun StorEdge A5000,Solstice SyMon, NFS, JumpStart, Sun VTS, OpenBoot, and AnswerBook are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and othercountries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Productsbearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering effortsof Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xeroxto the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s writtenlicense agreements.

U.S. Government approval required when exporting the product.

RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Govt is subject to restrictions of FAR 52.227-14(g) (2)(6/87) and FAR 52.227-19(6/87), or DFAR252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANYIMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THEEXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Sun Enterprise Cluster Administration – Veritas Volume Manager iiiCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Course Contents

About This Course .......................................................................................................Preface-1Course Goal .................................................................................................................................. Preface-2Course Overview ......................................................................................................................... Preface-3Course Map ................................................................................................................................... Preface-4Module-by-Module Overview ................................................................................................... Preface-5Course Objectives ......................................................................................................................... Preface-7Skills Gained by Module ........................................................................................................... Preface-10Guidelines for Module Pacing ................................................................................................. Preface-11Topics Not Covered ................................................................................................................... Preface-13How Prepared Are You? ........................................................................................................... Preface-14Introductions .............................................................................................................................. Preface-15How to Use Course Materials .................................................................................................. Preface-16How to Use the Icons ................................................................................................................. Preface-17Typographical Conventions and Symbols ............................................................................. Preface-19

Sun Cluster Overview ............................................................................................................1-1Overview .................................................................................................................................................. 1-2Sun Cluster 2.2 New Features ............................................................................................................... 1-3Cluster Hardware Components ............................................................................................................ 1-4High Availability Features .................................................................................................................... 1-5High Availability Strategies .................................................................................................................. 1-6Cluster Configurations ........................................................................................................................... 1-7Sun Cluster Application Support ......................................................................................................... 1-8Logical Hosts ........................................................................................................................................... 1-9Fault Monitoring ................................................................................................................................... 1-10Failure Recovery Summary ................................................................................................................. 1-11


Sun Enterprise Cluster Administration – Veritas Volume Manager ivCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Check Your Progress ............................................................................................................................ 1-13Think Beyond ........................................................................................................................................ 1-15

Terminal Concentrator ..........................................................................................................2-1Module Overview ................................................................................................................................... 2-2Cluster Administration Interface .......................................................................................................... 2-3Terminal Concentrator Overview ........................................................................................................ 2-5Terminal Concentrator Setup ................................................................................................................ 2-7Terminal Concentrator Troubleshooting ........................................................................................... 2-10Exercise ................................................................................................................................................... 2-11Check Your Progress ............................................................................................................................ 2-12Think Beyond ........................................................................................................................................ 2-14

Administrative Workstation ................................................................................................3-1Overview .................................................................................................................................................. 3-2Cluster Software Summary ................................................................................................................... 3-3Sun Enterprise Cluster Software Summary ........................................................................................ 3-4Software Installation Program .............................................................................................................. 3-5Sun Cluster Installation Program Startup ........................................................................................... 3-6Administration Workstation Environment ......................................................................................... 3-9Cluster Administration Tools Configuration .................................................................................... 3-11Cluster Control Panel ........................................................................................................................... 3-13Cluster Administration Tools .............................................................................................................. 3-15Exercise ................................................................................................................................................... 3-17Check Your Progress ............................................................................................................................ 3-18Think Beyond ........................................................................................................................................ 3-19

Preinstallation Configuration ..............................................................................................4-1Module Overview ................................................................................................................................... 4-2Cluster Topologies .................................................................................................................................. 4-3


Sun Enterprise Cluster Administration – Veritas Volume Manager vCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Cluster Quorum Devices ........................................................................................................................ 4-9Cluster Interconnect System Overview ............................................................................................. 4-14Cluster Interconnect System Configuration ...................................................................................... 4-16Public Network Management ............................................................................................................. 4-23Shared CCD Volume ............................................................................................................................ 4-26Cluster Configuration Information .................................................................................................... 4-28Storage Array Firmware Upgrades .................................................................................................... 4-33Exercise ................................................................................................................................................... 4-34Check Your Progress ............................................................................................................................ 4-35Think Beyond ........................................................................................................................................ 4-36

Cluster Host Software Installation .....................................................................................5-1Module Overview ................................................................................................................................... 5-2Sun Cluster Server Software Overview ............................................................................................... 5-3Sun Cluster Installation Overview ....................................................................................................... 5-5Sun Cluster Volume Managers ............................................................................................................. 5-6Sun Cluster Host System Configuration ............................................................................................. 5-8Sun Cluster Private Network Configuration ...................................................................................... 5-9Sun Cluster Public Network Configuration ...................................................................................... 5-11Sun Cluster Logical Host Configuration ........................................................................................... 5-13Data Protection Configuration ............................................................................................................ 5-15Application Configuration .................................................................................................................. 5-20Post-Installation Configuration ........................................................................................................... 5-22Exercise ................................................................................................................................................... 5-26Check Your Progress ............................................................................................................................ 5-27Think Beyond ........................................................................................................................................ 5-28

System Operation ...................................................................................................................6-1Module Overview ................................................................................................................................... 6-2Cluster Administration Tools ................................................................................................................ 6-3


Sun Enterprise Cluster Administration – Veritas Volume Manager viCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Basic Cluster Control (scadmin ) ........................................................................................................... 6-5Cluster Control Panel ............................................................................................................................. 6-6The hastat Command ........................................................................................................................... 6-8Sun Cluster Manager Overview ......................................................................................................... 6-15Sun Cluster Manager Displays ........................................................................................................... 6-17Sun Cluster SNMP Agent .................................................................................................................... 6-23Exercise ................................................................................................................................................... 6-25Check Your Progress ............................................................................................................................ 6-26Think Beyond ........................................................................................................................................ 6-27

Volume Management UsingCVM and SSVM ......................................................................................................................7-1

Module Overview ................................................................................................................................... 7-2CVM and SSVM Disk Space Management .......................................................................................... 7-3CVM and SSVM Initialization ............................................................................................................... 7-5CVM and SSVM Encapsulation ............................................................................................................ 7-6CVM and SSVM Disk Grouping ........................................................................................................... 7-8Volume Manager Status ....................................................................................................................... 7-11Optimizing Recovery Times ................................................................................................................ 7-13CVM and SSVM Post-Installation ...................................................................................................... 7-14Check Your Progress ............................................................................................................................ 7-16Think Beyond ........................................................................................................................................ 7-17

Cluster Configuration Database ..........................................................................................8-1Module Overview ................................................................................................................................... 8-2Cluster Configuration Information ...................................................................................................... 8-3Cluster Database Consistency ............................................................................................................... 8-4Shared CCD Volume .............................................................................................................................. 8-6CCD Administration .............................................................................................................................. 8-9Exercise ................................................................................................................................................... 8-10


Sun Enterprise Cluster Administration – Veritas Volume Manager viiCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Check Your Progress ............................................................................................................................ 8-11Think Beyond ........................................................................................................................................ 8-12

Public Network Management ...............................................................................................9-1Module Overview ................................................................................................................................... 9-2PNM Overview ....................................................................................................................................... 9-3The Network Monitoring Process ........................................................................................................ 9-5How PNM Works ................................................................................................................................... 9-6PNM Monitoring Routines .................................................................................................................... 9-7The pnmset Command ........................................................................................................................... 9-8Other PNM Commands ....................................................................................................................... 9-10Exercise ................................................................................................................................................... 9-11Check Your Progress ............................................................................................................................ 9-12Think Beyond ........................................................................................................................................ 9-13

Logical Hosts ........................................................................................................................10-1Module Overview ................................................................................................................................. 10-2Logical Hosts ......................................................................................................................................... 10-3Configuring a Logical Host ................................................................................................................. 10-5Logical Host Variations ........................................................................................................................ 10-6Administrative File System Overview ............................................................................................... 10-7Creating the Administrative File System .......................................................................................... 10-8Logical Host File Systems .................................................................................................................... 10-9Logical Host Control .......................................................................................................................... 10-10Exercise ................................................................................................................................................. 10-11Check Your Progress .......................................................................................................................... 10-12Think Beyond ...................................................................................................................................... 10-13

The HA-NFS Data Service ...................................................................................................11-1Module Overview ................................................................................................................................. 11-2


Sun Enterprise Cluster Administration – Veritas Volume Manager viiiCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Objectives ............................................................................................................................................... 11-3HA-NFS Overview ................................................................................................................................ 11-4HA-NFS Data Service ........................................................................................................................... 11-5Start NFS Methods ................................................................................................................................ 11-6Stop NFS Methods ................................................................................................................................ 11-7HA-NFS Fault Monitoring ................................................................................................................... 11-8Fault Probes ........................................................................................................................................... 11-9Local Fault Probes ............................................................................................................................... 11-10Remote Fault Probes ........................................................................................................................... 11-11Giveaway and Takeaway Process .................................................................................................... 11-12Processes Related to NFS Fault Monitoring .................................................................................... 11-14HA-NFS Support Files ........................................................................................................................ 11-15Using the hareg Command ............................................................................................................... 11-16File Locking Recovery ........................................................................................................................ 11-17Exercise ................................................................................................................................................. 11-18Check Your Progress .......................................................................................................................... 11-19Think Beyond ...................................................................................................................................... 11-20

System Recovery ...................................................................................................................12-1Module Overview ................................................................................................................................. 12-2Sun Cluster Reconfiguration Control ................................................................................................. 12-3Sun Cluster Failfast Driver .................................................................................................................. 12-5Sun Cluster Reconfiguration Sequence .............................................................................................. 12-7Cluster Interconnect Failures ............................................................................................................ 12-11Two-Node Partitioned Cluster Failure ............................................................................................ 12-12Logical Host Reconfiguration ........................................................................................................... 12-13Exercise ................................................................................................................................................. 12-15Check Your Progress .......................................................................................................................... 12-16Think Beyond ...................................................................................................................................... 12-17


Sun Enterprise Cluster Administration – Veritas Volume Manager ixCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Sun ClusterHigh Availability Data Service API .................................................................................13-1

Module Overview ................................................................................................................................. 13-2Overview ................................................................................................................................................ 13-3Data Service Requirements .................................................................................................................. 13-4Reconfiguration Overview .................................................................................................................. 13-5Data Service Methods ........................................................................................................................... 13-6Giveaway and Takeaway ..................................................................................................................... 13-7START and STOP Method Examples ................................................................................................. 13-8Data Service Dependencies ............................................................................................................... 13-10The haget Command ......................................................................................................................... 13-11The hactl Command ......................................................................................................................... 13-12The halockrun Command ................................................................................................................ 13-13The hatimerun Command ................................................................................................................ 13-14Exercise ................................................................................................................................................. 13-15Check Your Progress .......................................................................................................................... 13-16Think Beyond ...................................................................................................................................... 13-17

Highly Available DBMS ......................................................................................................14-1Module Overview ................................................................................................................................. 14-2Sun Cluster HA-DBMS Overview ...................................................................................................... 14-3Typical HA-DBMS Configuration ...................................................................................................... 14-4Configuring and Starting HA-DBMS ................................................................................................. 14-5Stopping and Unconfiguring

HA-DBMS ........................................................................................................................................... 14-6The HA-DBMS Start Methods ............................................................................................................. 14-7The HA-DBMS Stop and Abort Methods .......................................................................................... 14-8HA-DBMS Fault Monitoring ............................................................................................................... 14-9Configuring HA-DBMS for High Availability ............................................................................... 14-10Configuration Overview .................................................................................................................... 14-11


Sun Enterprise Cluster Administration – Veritas Volume Manager xCopyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Oracle Installation Preparation ......................................................................................................... 14-12Sybase Installation Preparation ........................................................................................................ 14-13Informix Installation Preparation ..................................................................................................... 14-14Preparing the Logical Host ................................................................................................................ 14-15HA-DBMS Control .............................................................................................................................. 14-16HA-DBMS Client Overview .............................................................................................................. 14-17HA-DBMS Recovery ........................................................................................................................... 14-18HA-DBMS Configuration Files ......................................................................................................... 14-19Exercise ................................................................................................................................................. 14-20Check Your Progress .......................................................................................................................... 14-21Think Beyond ...................................................................................................................................... 14-22



Preface

About This Course


Sun Enterprise Cluster Administration – Veritas Volume Manager Preface, slide 2 of 18Copyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999, Revision A

Course Goal

This course provides you with knowledge and skills to installand administer a Sun Enterprise™ Cluster system runningSun Cluster 2.2 software.



Course Overview

• Sun Enterprise Cluster hardware configuration

• Sun Cluster software installation

• Sun Cluster software configuration

• Sun Enterprise Cluster operation

• System failure recovery


Sun Enterprise Cluster Administration – Veritas Volume Manager Preface, slide 4 of 18Copyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services, October 1999, Revision A

Course MapIntroduction

Installation

Operation

Logical Host Configuration

Recovery

High Availability

Sun ClusterOverview

TerminalConcentrator

AdministrationWorkstationInstallation

PreinstallationConfiguration

Cluster HostSoftware

Installation

SystemOperation

VolumeManagement

CVM and SSVM

ClusterConfiguration

Database

Public NetworkManagement

Logical Hosts The HA-NFSData Service

SystemRecovery

Sun ClusterHA-API

HighlyAvailable

DBMS



Module-by-Module Overview

• Module 1 – "Sun Cluster Overview"

• Module 2 – "Terminal Concentrator"

• Module 3 – "Administration Workstation Installation"

• Module 4 – "Preinstallation Configuration"

• Module 5 – "Cluster Host Software Installation"

• Module 6 – "System Operation"

• Module 7 – "Volume Management Using CVM andSSVM"



Module-by-Module Overview

• Module 8 - "Cluster Configuration Database"

• Module 9 - "Public Network Management"

• Module 10 - "Logical Hosts"

• Module 11 - "The HA-NFS Data Service"

• Module 12 - "System Recovery"

• Module 13 - "Sun Cluster High Availability Data ServiceAPI"

• Module 14 - "Highly Available DBMS"



Course Objectives

• Describe major Sun Cluster components and functions

• Verify system cabling

• Configure the Terminal Concentrator for properoperation

• Install, remove, and update Sun Cluster software

• Troubleshoot software installation and configurationerrors



Course Objectives

• Configure environmental variables for correct SunCluster operation

• Use the Sun Cluster administration tools

• Initialize one of the supported volume managers

• Describe the differences between the supported volumemanagers

• Prepare the Public Network Management failoverenvironment



Course Objectives

• Create and configure logical hosts

• Install and configure highly available data services

• Describe the Sun Cluster failure recovery mechanisms

• Identify and recover from selected Sun Cluster failures



Skills Gained by Module

Meaning of:

• Black boxes

• Gray boxes

Module

Skills Gained 1 2 3 4

Skill or Objective 1






Guidelines for Module PacingModule Day 1 Day 2 Day 3 Day 4 Day 5

About This Course A.M.

Product Introduction A.M.

Terminal Concentrator A.M./P.M.

Administration Workstation Installation P.M.

Pre-Installation Configuration A.M.

Cluster Host Software Installation A.M./P.M.

System Operation P.M.

Volume Management, CVM/SSVM A.M.

Cluster Configuration Database P.M.

Public Network Management P.M.

Logical Hosts A.M.

HA-NFS Data Service A.MP.M.

System Recovery PM

Sun Enterprise Cluster HA-API A.M.

HA-DBMS A.M./P.M.



Topics Not Covered

• Database management

• Network administration

• Solaris™ Operating Environment administration

• Performance and tuning

• Disk storage management



How Prepared Are You?

• Can you explain virtual volume managementterminology, such as mirroring, striping, concatenation,volumes, and mirror synchronization?

• Can you perform basic Solaris system administrationtasks, such as using tar and ufsdump commands,creating user accounts, formatting disk drives, using vi ,installing the operating system, installing patches, andadding packages?

• Do you have prior experience with Sun hardware andthe OpenBoot™ PROM technology?

• Are you familiar with general computer hardware,electro-static precautions, and safe handling practices?



Introductions

• Name

• Company affiliation

• Title, function, and job responsibility

• Clustered systems experience

• Reasons for enrolling in this course

• Expectations for this course



How to Use Course Materials

• Course Map

• Relevance

• Overhead image

• Lecture

• Exercise

• Check Your Progress

• Think Beyond



How to Use the Icons

Additional resources

Discussion

Exercise objective



Caution

Warning

!



Typographical Conventions andSymbols

• Courier is used for the names of command, files, anddirectories, as well as on-screen computer output.

• Courier bold is used for characters and numbers thatyou type.

• Courier italic is used for variables and command-line placeholders that are replaced with a real name orvalue.

• Palatino italics is used for book titles, new words orterms, or words that are emphasized.



Module 1

Sun Cluster Overview


Sun Enterprise Cluster Administration – Veritas Volume Manager Module 1, slide 2 of 15Copyright 1999 Sun Microsystems, Inc. All Rights Reserved. Enterprise Services October 1999 Revision A

Overview

• Objectives

• Relevance



Sun Cluster 2.2 New Features

The Sun Cluster 2.2 software release has the following newfeatures:

• Sun Cluster 2.2 is now fully internationalized and Y2Kcompliant

• Support for Solstice DiskSuite has been added

• Solaris 7 Operating Environment is now supported

• The installation procedures have been changed

• Licensing is much simpler

• There is a new cluster management tool Sun ClusterManager



Cluster Hardware Components

Public network

Administration Workstation

Terminal Concentrator

Mass storage

Cluster hostAA

Mass storage

Cluster host

Private network



High Availability Features

Several software and hardware features contribute to thecluster availability. They include:

• High availability hardware design

• Sun Cluster software

• Software RAID technology

• Hardware RAID technology

• Year 2000 compliance



High Availability Strategies

Node 1Node 0

Mass storage Mass storage

Volumemanager

Application

Primarydata

Mirrordata

Volumemanager

Application

User

hme0 hme1hme0 hme1

Public network

Private networks



Cluster Configurations

Node 1Node 0

Mass storage

Volumemanager

Database data

Volumemanager

Databaseinstance 0

Databaseinstance 1

Parallel Database

Node 1Node 0


Volumemanager

Application B

Volumemanager

Application A

Application Adata

Application Bdata

Highly Available Data Service



Sun Cluster Application Support

The Sun Cluster software framework provides support for thefollowing application categories:

• Highly available databases

• Highly available data services

• Parallel databases



Logical Hosts

Disk group

ccd.database

Node 0

Network Access

Primary: Node 0Backup: Node 1

Application

Node 1

ccd.database

Lhost_1

information


ifconfig

import

Diskgroup

start



Fault Monitoring

Data Service

Localfaultmonitor

Remotefaultmonitor

Networkdriver

Networkdriver

Private networks

Heartbeats

SMA SMA

Node 0 Node 1

check

Node 0 Node 1

Data Service Fault Monitoring

Cluster Fault Monitoring

PNM PNM

CMM CMM

Failfast Failfast

check

Public network



Failure Recovery Summary

• Individual disk drive failure

• Fibre Channel or array controller failure

• Cluster interconnect failure

• Node failure

• Critical process or daemon failure

• Cluster configuration database file inconsistency



Failure Recovery Summary

Node 0 Node 1

Networkdriver

Networkdriver Private networks

CMM

RDBMS

SMA

Diskmanagement

Fiber-opticchannels

Storage array

RDBMS

Diskmanagement

Fiber-opticchannels

Storage array

failfastdriver (ff )

failfastdriver (ff )

CDB

CCD

CMM

SMA

CDB

CCD

Heartbeat

Heartbeat

Consistency

Consistency



Check Your Progress

• List the hardware elements that comprise a basic SunEnterprise Cluster system

• List the hardware and software components thatcontribute to the availability of a Sun Enterprise Clustersystem

• List the types of redundancy that contribute to theavailability of a Sun Enterprise Cluster system.

• Identify the configuration differences between a highavailability cluster and a parallel database cluster



Check Your Progress

• Explain the purpose of logical host definitions in theSun Enterprise Cluster environment

• Describe the purpose of the cluster configurationdatabases

• Explain the purpose of each of the Sun EnterpriseCluster fault monitoring mechanisms.



Think Beyond

What are some of the most common problems encounteredduring cluster installation?

How does a cluster installation proceed? What do you need todo first?

Do you need to be a database expert to administer a SunEnterprise Cluster system?



Module 2




Module Overview

• Objectives

• Relevance



Cluster Administration Interface

• Administration workstation

• Cluster administration tools

• Terminal Concentrator

• Cluster host serial port connections



Cluster Administration Interface

Node 0Serial port A

Setup device

Node 1


Networkinterface Serial ports

1 2 4 6 753 8

Setupport

Administration workstation

AdministrationTools

Network



Terminal Concentrator Overview

• Resident PROM-based operating system

• Special setup port

• Setup programs



Terminal Concentrator Overview

Node 0Serial port A

Setup device

Node 1


Networkinterface

Serial ports

Memory

PROMOperating system

Self-load at power-on

OPER_52_ENET.SYS

Setupport 1 2 4 6 753 8



Terminal Concentrator Setup

• Connecting to Port 1

• Enabling setup mode

• Setting IP address

• Setting load sequence

• Specifying operating system image

• Setting serial port variables




1 2 3 4 5 6 7 8

STATUS

1 2 3 4 5 6 7 8 POWER UNIT NET ATTN LOAD ACTIVE

Power indicator Test button




admin-ws# telnet terminal_concentrator_nameTrying terminal concentrator IP addressConnected to sec-tc.Escape character is '̂ ]'.Rotaries Defined: cli -Enter Annex port name or number: cliAnnex Command Line Interpreter * Copyright 1991 Xylogics, Inc.annex: suPassword: type the passwordannex# adminAnnex administration MICRO-XL-UX R7.0.1, 8 portsadmin: show port=1 type mode imask_7bitsPort 1:type: hardwired mode: cli imask_7bits: Yadmin: set port=1 type hardwired mode cli imask_7bits Yadmin: set port=2-8 type dial_in mode slave imask_7bits Yadmin: quitannex# bootbootfile: <CR>warning: <CR>



Terminal ConcentratorTroubleshooting

• Use the telnet command to manually connect to anode

• Use the telnet command to abort a node

• Use the terminal concentrator help command

• Use the who and reset commands to free a port

• Use the erase command to reset the password



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Describe the Sun Enterprise Cluster administrativeinterface

• Explain the TC hardware configuration

• Verify the correct TC cabling

• Configure the TC IP address

• Configure the TC to self-load

• Verify the TC port settings



Check Your Progress

• Verify that the TC is functional

• Use the terminal concentrator help , who, and hangupcommands

• Describe the purpose of the telnet send brkcommand



Think Beyond

Is there a significant danger if the TC port variables are not setcorrectly?

Is the Terminal Concentrator a single point of failure? Whatwould happen if it failed?



Module 3

Administrative Workstation



Overview

• Objectives

• Relevance



Cluster Software Summary

scinstall

Software

(installation script)

Sun_Cluster_2_2

Sol_7 Sol_2.6

Product Tools ProductToolsSoftware

ClusterVolume Manager

Volume Manager

Volume Manager

Sun StorEdge

Solstice DiskSuite



Sun Enterprise ClusterSoftware Summary

Network

Administrationworkstation

hardware

Node 0system

Node 1systemhardware

Private disk

Solaris 2.6/7

Sun Clusterclient software

Solaris 2.6/7

Sun Clusterserver software

Volumemanagement

Private disk

Solaris 2.6/7


Volumemanagement

Disk storage array Disk storage array



Software Installation Program

• The scinstall command installs the Sun Clustersoftware

• Script runs interactively and prompts user



Sun Cluster Installation ProgramStartup

• Event sequence during Sun Cluster packageadministration tool startup phase:

• SUNWscins package automatically started

• Sun Cluster software status assessed

• Package administration category selected

• Sun Cluster package source identified

• Package installation mode selected




Choose one:1) UpgradeUpgrade to Sun Cluster 2.2 Server packages2) ServerInstall the Sun Cluster packages needed on a server3) ClientInstall the admin tools needed on an admin workstation4) Server and ClientInstall both Client and Server packages

5) CloseExit this Menu6) QuitQuit the Program

Enter the number of the package set [6]: 3




Installing Client packages

Installing the following packages: SUNWscch SUNWccon SUNWccp SUNWcsnmp SUNWscsdb

>>>> Warning <<<< The installation process will run several scripts as root. In addition, it may installsetUID programs. If you choose automatic mode, the installation of the chosen packageswill proceed without any user interaction.If you wish to manually control the installprocess you must choose the manual installation option.

Choices:manual Interactively install each packageautomatic Install the selected packages with no user interaction.

In addition, the following commands are supported: list Show a list of the packages to be installed help Show this command summary close Return to previous menu quit Quit the program

Install mode [manual automatic] [automatic]: automatic



Administration WorkstationEnvironment

• Environmental changes after client softwareinstallation:

• New search and man page paths

• Host name resolution changes

• Remote login control

• Remote display enabling

• rsh and rcp access control



AdministrationWorkstation Environment

Access allowed

Access denied

Passwordprompt

No

Yes

Yes

Yes

No

rlogin

rcp

host X

No

No

Command?

Login prompt

rsh

Passwordcorrect?

No Yes

host A

Is user JD in/etc/passwd?

Is user JDa superuser?

Is host A in/etc/hosts.equiv?

Is host A in$HOME/.rhosts?

Yes

User JD wants to rlogin, rcp, or rsh to host X



Cluster Administration ToolsConfiguration

• Configuration information required for clusteradministration tools:

• Cluster name

• Each physical node name and IP address

• Terminal concentrator name and IP address

• TC port number used for cluster attachment



Cluster AdministrationTools Configuration


Administrationtools

Network

Serial port A

Node 0 Node 2 Node 3Node 1

Terminal concentrator

Networkinterface

Serial ports

1 6 7 84 532



Cluster Control Panel

• Cluster Control Panel

• Cluster Console

• Cluster Help Tool

• Cluster Manager



Cluster Administration Tools

Node 0 window

Common window

Node 1 window

Node 2 window Node 3 window



Cluster AdministrationTools



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Summarize the Sun Cluster administrative workstationfunctions

• Use the scinstall script features

• Install the client software on the administrationworkstation

• Set up the administration workstation environment

• Configure the Sun Cluster administration tools



Think Beyond

What is the advantage of the /etc/clusters and/etc/serialports files?

Why is the Cluster SNMP agent installed on theadministrative workstation?

What is the impact on the cluster if the administrativeworkstation is not available? What would you do for backup?



Module 4

Preinstallation Configuration



Module Overview

• Objectives

• Relevance



Cluster Topologies

• Scalable

• Ring

• N+1

• Clustered pairs

• Shared-nothing



Cluster Topologies

Array Array Array Array

A B A B A B A B

Node 0 Node 1 Node 2 Node 3

CIS switchor hub



Cluster Topologies

Node 0

Node 3

Node 2

Node 1CIS switch

or hub



Cluster Topologies

Array

A B

Array

A B

Array

A B

CIS switchor hub




Cluster Topologies


CIS switchor hub

Array Array Array Array

A B A B A B A B



Cluster Topologies


A5000 storagearray

a0 a1 b0 b1

CIS switchor hub



Cluster Quorum Devices

• Basic Quorum Device Usage

• Number of Quorum Devices Required




Node 0 Node 1

Disk storage array

QA B

Clusterinterconnect

Reserve Reserve

Database

Quorum= c2t0d0

Quorum= c2t0d0

Database

Quorum= c3t0d0

Database




Node 0 Node 1

Disk storage array

Database

Quorum= c2t0d0

QA B

Quorum= WWN

Database

Quorum= WWN

Database

Reserve Reserve

Controller

Clusterinterconnect




Node 0

A B

Node 1 Node 2

Resource1

A B

Resource2

A B

Resource3




TC

Direct attached storage

Ethernet

Node 3interconnectfailure

HUB

Node 1 Node 2 Node 3

62 3 4 5

Abort

Lock

Node 0



Cluster Interconnect System Overview

• Interconnect Types

• Interconnect Configurations



Cluster Interconnect System Overview

Node 0 Node 1

System board

System board

System board

System board

hme0

hme1 hme1

hme0



Cluster Interconnect SystemConfiguration

• Primary and backup interface identification

• Primary and backup interface connection

• Point-to-point connections

• SCI high-speed connections

• SCI card identification

• SCI card scrubber jumpers




204.152.65.1

204.152.65.17

204.152.65.2

204.152.65.18

204.152.65.3

204.152.65.19

204.152.65.4

204.152.65.20

204.152.65.33

204.152.65.34

204.152.65.35

204.152.65.36

hme 0, scid 0

hme 1, scid 1

hme 0, scid 0

hme 1, scid 1

hme 0, scid 0

hme 1, scid 1

hme 0, scid 0

hme 1, scid 1

First Node

Second Node

Third Node

Fourth Node




Node 0 Node 1

System board

System board

System board

System board

hme0

scid0

hme1

scid1

hme1

scid1

hme0

scid0

oror

or or

primary

backup




Node 0 Node 1

Node 2 Node 3

Switch 0

Switch 1

scid0

scid1

System board

System board

scid0

scid1

System board

System board

System board

System boardscid1

scid0

System board

System boardscid1

scid00 1

2 3

0 1

2 3




Scrubber jumper

On

Off




Node 0Node 1

Scrubber on

Scrubber on

Scrubber off

Scrubber off




Node 0 Node 1

Node 2 Node 3

Hub 0

Hub 1

hme0

hme1

System board

System board

hme0

hme1

System board

System board

System board

System boardhme1

hme0

System board

System boardhme1

hme0



Public Network Management

• Adapter failover

• IP address failover

• Continuous fault monitoring

• Backup group configuration




hme0

Network

Node 0

ccd

NAFO groupIP address

Node 1

hme1

pnmd

Primary Backup

nafo7 group

Monitorprimary

/etc/pnmconfig

backup

NAFO groupconfiguration

hme0 hme1

Primary Backup

nafo12 group

ifconfig




# /opt/SUNWpnm/bin/pnmset

In the following you will be prompted to do configuration for network adapter failover

do you want to continue...[y/n]: y

How many PNM backup groups on the host: 2

Enter backup group number: 0Please enter all network adapters under nafo0qe1 qe0

Enter backup group number: 1Please enter all network adapters under nafo1hme0



Shared CCD Volume

• Guarantee majority opinion about CCD consistency

• Volume creation



Shared CCD Volume

Node 1Node 0

Mass storage

I/O interfaces I/O interfaces

Mass storage

ccd primary ccd mirror

ccd.database ccd.database



Cluster Configuration Information

General cluster configuration information can be gatheredusing several commands including:

• The prtdiag command

• The finddevices command

• The luxadm command




# /usr/platform/sun4u/sbin/prtdiagSystem Configuration: Sun Microsystems sun4u 8-slot Sun Enterprise 4000/5000System clock frequency: 84 MHzMemory size: 512Mb

========================= CPUs =========================

Run Ecache CPU CPUBrd CPU Module MHz MB Impl. Mask--- --- ------- ----- ------ ------ ---- 0 0 0 168 0.5 US-I 2.2 0 1 1 168 0.5 US-I 2.2 2 4 0 168 0.5 US-I 2.2 2 5 1 168 0.5 US-I 2.2

========================= Memory =========================

Intrlv. Intrlv.Brd Bank MB Status Condition Speed Factor With--- ----- ---- ------- ---------- ----- ------- ------- 0 0 256 Active OK 60ns 2-way A 2 0 256 Active OK 60ns 2-way A




========================= IO Cards =========================

Bus FreqBrd Type MHz Slot Name Model--- ---- ---- ---- --------------------- --------------1 SBus 25 0 DOLPHIN,sci1 SBus 25 1 qec/be (network) SUNW,270-24501 SBus 25 2 QLGC,isp/sd (block) QLGC,ISP1000U1 SBus 25 3 SUNW,hme1 SBus 25 3 SUNW,fas/sd (block)1 SBus 25 13 SUNW,soc/SUNW,pln 501-2069

Detached Boards===============Slot State Type Info---- -------- ----- -----------------------------7 disabled disk Disk 0: Target: 14 Disk 1: Target: 15




SBus 1

TPE

SBuscard

SBuscard

SBuscard

FEPSSOC

SBus 0

slot 0slot 1slot 2slot 13 (d) slot 3

Fast/WideSCSI10/100

FCOM FCOM




# /opt/SUNWcluster/bin/finddevicesc2:00000078BF60c3:00000078B12Dc4:00000078BF9E

# luxadm probeFoundSENA Name:d Node WWN:5080020000011df0 Logical Path:/dev/es/ses0 Logical Path:/dev/es/ses1SENA Name:a Node WWN:50800200000291d8 Logical Path:/dev/es/ses2 Logical Path:/dev/es/ses3



Storage Array Firmware Upgrades

When considering storage array firmware upgrades, youmust be aware of the following points:

• Firmware patches change both software and firmware

• You should not make large firmware revision changes

• Several other patches might be necessary

• The patch README notes might contain criticalinformation

• You should consider getting qualified assistance



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Configure any supported cluster topology

• List the appropriate applications for each topology

• Configure the cluster interconnect system

• Explain the need for simple quorum device

• Estimate the number of quorum devices needed foreach cluster topology

• Describe the purpose of the public network monitorfeature

• Describe the purpose of the mirrored ccd volume

• Explain the purpose of the TC node locking port



Think Beyond

What additional preparation might be necessary beforeinstalling the Sun Cluster host software?



Module 5

Cluster Host Software Installation



Module Overview

• Objectives

• Relevance



Sun Cluster Server Software Overview

scinstall

Software

Sun_Cluster_2_2

Sol_7 Sol_2.6

Product Tools ProductToolsSoftware

(Installation script)



Sun Cluster ServerSoftware Overview

Network

Administrationworkstation

hardware

Node 0system

Node 1systemhardware

Private disk

Solaris 2.6/7

Sun Clusterclient software

Solaris 2.6/7


Volumemanagement

Private disk

Solaris 2.6/7


Volumemanagement

Disk storage array Disk storage array



Sun Cluster Installation Overview

• Configuration during Sun Cluster server softwareinstallation:

• Target volume manager

• Cluster host system names and configuration

• Private and public networks

• Logical hosts

• Data protection configuration

• Data services desired



Sun Cluster Volume Managers

• Cluster Volume Manager

• Sun StorEdge Volume Manager

• Solstice DiskSuite™



Sun Cluster Volume Managers

Volume Manager Selection

Please choose the Volume Manager that will be usedon this node:

1) Cluster Volume Manager (CVM)2) Sun Storage Volume Manager (SSVM)3) Solstice DiskSuite (SDS)

Choose the Volume Manager: 3

Installing Solstice DiskSuite support packages.Installing “SUNWdid” ... doneInstalling “SUNWmdm” ... done

---------WARNING---------Solstice DiskSuite (SDS) will need to be installed before the cluster can bestarted.

<<Press return to continue>>



Sun Cluster Host SystemConfiguration

• Cluster name

• Nodes:

• Each name

• Potential number

• Active number



Sun Cluster Private NetworkConfiguration

• Scalable Coherent Interface

• Ethernet (100BaseT)



Sun Cluster Private NetworkConfiguration

What is the hostname of node 0 [node0]? phys-hahost1

What is phys-hahost1's first private network interface [hme0]? hme0

What is phys-hahost1's second private network interface [hme1]? hme1

You will now be prompted for Ethernet addresses ofthe host. There is only one Ethernet address for each host regardless of thenumber of interfaces a host has. You can get this information in one of severalways:

1. use the 'banner' command at the ok prompt,2. use the 'ifconfig -a' command (need to be root),3. use ping, arp and grep commands. ('ping exxon; arp -a | grep exxon')

Ethernet addresses are given as six hexadecimal bytes separated by colons.ie, 01:23:45:67:89:ab

What is phys-hahost1's ethernet address? 01:23:45:67:89:ab

What is the hostname of node 1 [node1]?



Sun Cluster Public NetworkConfiguration

• Controller and name of each primary network

• Controller and name of each secondary network



Sun Cluster PublicNetwork Configuration

What is the primary public network controller for“phys-hahost1”? hme2What is the primary public network controller for“phys-hahost2”? hme2Does the cluster serve any secondary public subnets(yes/no) [no]? y

Please enter a unique name for each of these additionalsubnets:

Subnet name (̂ D to finish): sc-cluster-net1 Subnet name (̂ D to finish): sc-cluster-net2 Subnet name (̂ D to finish): ^D

The list of secondary public subnets is: sc-cluster-net1 sc-cluster-net2Is this list correct (yes/no) [yes]?

For subnet “sc-cluster-pub1” ...What network controller is used for "phys-hahost1"? qe0What network controller is used for "phys-hahost2"? qe0

For subnet "sc-cluster-pub2" ...What network controller is used for "phys-hahost1"? qe1What network controller is used for "phys-hahost2"? qe1

Initialize NAFO on "phys-hahost1" with one ctlr per group(yes/no) [yes]? y



Sun Cluster Logical HostConfiguration

• For each logical host:

• Name

• Master system

• Backup system

• Network access name

• Associated disk group



Sun Cluster Logical HostConfiguration

Will this cluster support any HA data services(yes/no) [yes]? yesOkay to set up the logical hosts for those HA services now(yes/no) [yes]? yesEnter the list of logical hosts you want to add:

Logical host (̂ D to finish): hahost1 Logical host (̂ D to finish): ^D

The list of logical hosts is: hahost1

Is this list correct (yes/no) [yes]? y

What is the name of the default master for “hahost1”? phys-hahost1

Enter a list of other nodes capable of mastering “hahost1”: Node name: phys-hahost2 Node name (̂ D to finish): ^D

The list that you entered is: phys-hahost1 phys-hahost2

Is this list correct (yes/no) [yes]? yEnable automatic failback for “hahost1” (yes/no) [no]? yWhat is the net name for “hahost1” on subnet “sc-cluster-net1”? hahost1-pub1What is the net name for “hahost1” on subnet “sc-cluster-net2”? hahost1-pub2Disk group name for logical host “hahost1” [hahost1]?Is it okay to add logical host “hahost1” now (yes/no)



Data Protection Configuration

• Failure fencing

• Node locking

• Quorum device

• Partitioned cluster control




TC1 2 3 4 5 6

Direct attached storage


Ethernettelenet tc_concentrator 5006




Getting device information for reachable nodes in the cluster.This may take a few seconds to a few minutes...doneSelect quorum device for the following nodes:0 (phys-hahost1)and1 (phys-hahost2)

1) SSA:000000779A162) SSA:0000007414303) DISK:c0t1d0s2:01799413Quorum device: 1...SSA with WWN 000000779A16 has been chosen as the quorum device.

Finished Quorum Selection




Node 0

A B

Node 1 Node 2

Resource1

A B

Resource2

A B

Resource3




In case the cluster partitions into subsets, which subset should stay up? ask) the system will always ask the operator. select) automatic selection of which subset should stay up.

Please enter your choice (ask|select) [ask]: selectYou have a choice of two policies:

lowest -- The subset containing the node with the lowest node ID value automaticallybecomes the new cluster. All other subsets must be manually aborted.

highest -- The subset containing the node with the highest node ID value automaticallybecomes the new cluster. All other subsets must be manually aborted.

Select the selection policy for handling partitions (lowest|highest) [lowest]: highest



Application Configuration

• Highly available data services

• Highly available databases

• Parallel databases



Application Configuration

==== Select Data Services Menu ====================

Please select which of the following data services are to be installed onto this cluster.Select singly, or in a space separated list.Note: HA-NFS and Informix Parallel Server (XPS) are installed automatically with theServer Framework.

You may de-select a data service by selecting it a second time.

Select DONE when finished selecting the configuration.

1) Sun Cluster HA for Oracle2) Sun Cluster HA for Informix3) Sun Cluster HA for Sybase4) Sun Cluster HA for Netscape5) Sun Cluster HA for Netscape LDAP6) Sun Cluster HA for Lotus7) Sun Cluster HA for Tivoli8) Sun Cluster HA for SAP9) Sun Cluster HA for DNS10) Sun Cluster for Oracle Parallel Server

INSTALL 11) No Data Services12) DONE

Choose a data service: 3



Post-Installation Configuration

• Verify installation

• Configure new software directory paths

• Complete SCI interconnect installation

• Install Sun Cluster patches




# scconf sc-cluster -p/etc/opt/SUNWcluster/conf/sc-cluster.cdbChecking node status...

Current Configuration for Cluster sc-cluster:

Hosts in cluster: phys-node0 phys-node1 phys-node2

Private Network Interfaces forphys-node0:be0 be1phys-node1:be0 be1phys-node2:be0 be1

Quorum Device Information

Logical Host Timeout Values :Step10 : 720Step11 : 720Logical Host : 180

Cluster TC/SSP Informationphys-node0 TC/SSP, port : 129.150.218.35, 2phys-node1 TC/SSP, port : 129.150.218.35, 3phys-node2 TC/SSP, port : 129.150.218.35, 4sc-cluster Locking TC/SSP, port : 129.150.218.35, 6




Cluster is configured as = SC

HOST 0 = sec-0HOST 1 = sec-1HOST 2 = _%sec-2HOST 3 = _%sec-3

Number of Switches in cluster = 0Number of Direct Links in cluster = 2Number of Rings in cluster = 0

host 0 :: adp 0 is connected to = link 0 :: endpt 0host 0 :: adp 1 is connected to = link 1 :: endpt 0

host 1 :: adp 0 is connected to = link 0 :: endpt 1host 1 :: adp 1 is connected to = link 1 :: endpt 1

Network IP address for Link 0 = 204.152.65Network IP address for Link 1 = 204.152.65

Netmask = f0




Cluster is configured as = SC

HOST 0 = sec-0HOST 1 = sec-1HOST 2 = sec-2

Number of Switches in cluster = 2Number of Direct Links in cluster = 0Number of Rings in cluster = 0

host 0 :: adp 0 is connected to = switch 0 :: port 0host 0 :: adp 1 is connected to = switch 1 :: port 0host 1 :: adp 0 is connected to = switch 0 :: port 1host 1 :: adp 1 is connected to = switch 1 :: port 1host 2 :: adp 0 is connected to = switch 0 :: port 2host 2 :: adp 1 is connected to = switch 1 :: port 2

Network IP address for Switch 0 = 204.152.65Network IP address for Switch 1 = 204.152.65

Netmask = f0



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Install the Sun Cluster host system software

• Correctly interpret configuration questions during SunCluster node software installation on the cluster hostsystems

• Perform post-installation configuration



Think Beyond

As you add additional nodes to the cluster, what might youneed to do on the existing nodes? Can you do this while thenodes are running?

What kinds of configuration changes need to be madesimultaneously on all nodes? How can you tell? What wouldhappen if you did not make them simultaneously?

What would happen if you did not specify any quorumdevices or use failure fencing?



Module 6

System Operation



Module Overview

• Objectives

• Relevance



Cluster Administration Tools

• Basic Cluster Control (scadmin )

• Cluster Control Panel

• hastat Command

• Sun Cluster Manager



Cluster AdministrationTools


Network


Node 0

Serial ports

Networkinterface

Sun Cluster Manager

Custer Control

# scadmin# hastat

Panel



Basic Cluster Control (scadmin )

• Initially starts cluster

• Activates additional cluster nodes

• Removes nodes from clustered operation




• Three cluster console tool variations

• Cluster help tool



The hastat Command

The hastat command provides a complete cluster statuslisting that includes:

• General cluster status

• Logical host configuration information

• Private network status

• Public network status

• Data service status

• Cluster error messages



The hastat Command

# hastatGetting Information from all the nodes ......

HIGH AVAILABILITY CONFIGURATION AND STATUS-------------------------------------------

LIST OF NODES CONFIGURED IN <sc-cluster> CLUSTERsc-node0 sc-node1 sc-node2

CURRENT MEMBERS OF THE CLUSTERsc-node0 is a cluster membersc-node1 is a cluster membersc-node2 is a cluster member

CONFIGURATION STATE OF THE CLUSTERConfiguration State on sc-node0: StableConfiguration State on sc-node1: StableConfiguration State on sc-node2: Stable

UPTIME OF NODES IN THE CLUSTER

uptime of sc-node0:12:47am up 1:38, 1 user,load average: 0.14, 0.12, 0.10





The hastat Command

LOGICAL HOSTS MASTERED BY THE CLUSTER MEMBERS

Logical Hosts Mastered on sc-node0:sc-dbms

Loghost Hosts for which sc-node0 is Backup Node:sc-nfs

Logical Hosts Mastered on sc-node1:sc-nfs

Loghost Hosts for which sc-node1 is Backup Node:sc-inetpro

Logical Hosts Mastered on sc-node2:sc-inetpro

Loghost Hosts for which sc-node2 is Backup Node:sc-dbms

LOGICAL HOSTS IN MAINTENANCE STATE

None



The hastat Command

STATUS OF PRIVATE NETS IN THE CLUSTER

Status of Interconnects on sc-node0: interconnect0: selected interconnect1: up Status of private nets on sc-node0: To sc-node0 - UP To sc-node1 - UP To sc-node2 - UP





The hastat Command

STATUS OF PUBLIC NETS IN THE CLUSTER

Status of Public Network On sc-node0:

bkggrp r_adp status fo_time live_adpnafo113 hme1 OK NEVER hme1







The hastat Command

STATUS OF SERVICES RUNNING ON LOGICAL HOSTS IN THE CLUSTER

Status Of Data Services Running On sc-node0 Data Service HA-SYBASE: No Status Method for Data Service dns

Data Service HA-NFS: On Logical Host sc-dbms: Ok


Data Service HA-NFS: On Logical Host sc-nfs: Ok


Data Service HA-NFS: On Logical Host sc-inetpro: Ok



The hastat Command

RECENT ERROR MESSAGES FROM THE CLUSTER

Recent Error Messages on sc-node0

Feb 2 00:24:20 sc-node0 unix: sbusmem51 at sbus3: SBus3 slot 0x3 offset 0x0Feb 2 00:24:20 sc-node0 unix: sbusmem51 is /sbus@7,0/sbusmem@3,0Feb 2 00:36:31 sc-node0 ID[SUNWcluster.ha.hareg.2004]: Service dns is registered


Feb 2 00:24:22 sc-node1 unix: sbusmem45 at sbus2: SBus2 slot 0xd offset 0x0Feb 2 00:24:22 sc-node1 unix: sbusmem45 is /sbus@6,0/sbusmem@d,0Feb 2 00:24:22 sc-node1 unix: sbusmem48 at sbus3: SBus3 slot 0x0 offset 0x0


Feb 2 00:27:05 sc-node2 unix: sbusmem13 at sbus0: SBus0 slot 0xd offset 0x0Feb 2 00:27:05 sc-node2 unix: sbusmem13 is /sbus@1f,0/sbusmem@d,0Feb 2 00:27:05 sc-node2 unix: sbusmem14 at sbus0: SBus0 slot 0xe offset 0x0



Sun Cluster Manager Overview

• Overall cluster status

• Software and hardware failure

• System log file viewer

• Hardware component displays



SunClusterManagerOverview



Sun Cluster Manager Displays

The Sun Cluster Manager tool has several interactivewindows that provide:

• Detailed cluster configuration information

• A log of significant cluster events

• A system error log filter

• A comprehensive Help tool



Sun Cluster ManagerDisplays



Sun Cluster SNMP Agent

• Management Information Base (MIB)

• SNMP Traps



Sun Cluster SNMP Agent

Trap No. Trap Name

0 sc:stopped

1 sc:aborted

4 sc:excluded

11 vm:down

21 db:up

31 vm_on_node:slave

100 SOCKET_ERROR:node_out_of_system_resources

106 UNREACHABLE_ERROR:node’s_mond_unreachable:network_problems ??

110 SHUTDOWN_ERROR:node’s_mond_shutdown

200 Fatal:super_monitor_daemon(smond)_exited!!



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Use the cluster administration tools

• Use the Sun Cluster Manager GUI

• Use the hastat status command

• List the SNMP features



Think Beyond

What would it be like to administer a cluster with 16 nodesand 200 storage arrays?

What would it be like to administer a cluster with each clustermember located in a different city?

How does SNMP in general interact with the clusterenvironment?



Module 7

Volume Management UsingCVM and SSVM



Module Overview

• Objectives

• Relevance



CVM and SSVM Disk SpaceManagement

• Space management at the block level

• Eliminates the UNIX® partition limit

• Equate disk regions with virtual volume structures



CVM and SSVM Disk SpaceManagement

Blocks 1000 – 3000

Blocks 5501 – 10000

Blocks 3001 – 5500

Blocks 10001 – 12000

Physical disk drive

Volume 01

Volume 02

Volume 03

Slice 4



CVM and SSVM Initialization

Public region

Private region

Configuration andmanagement information

Data Storage



CVM and SSVM Encapsulation

• Encapsulation preserves existing data

• The system boot disk can be encapsulated

• There is a preferred configuration

• Some free space is required

• Only standard partitions should be present

• Can be returned to a sliced configuration

• Boot disk mirrors are structured differently

• Mirrors cannot be returned to a sliced configuration



CVM and SSVM Encapsulation

c0

c2SOC

SCSI

SCSIc1

rootvol rootmirror

rootdgdisk group

Storage Array

newdgdisk group



CVM and SSVM Disk Grouping

• CVM and SSVM create disk groups

• CVM disk groups are shared between systems

• The disk group is owned by the cluster

• All attached systems have simultaneous access

• Access must be arbitrated

• SSVM disk groups are owned by only one system

• Access from only one system at a time

• Ownership can be transferred between systems




Storage array

Volume

Disk group

VolumeVolume

A B

The disk groupis owned by thecluster.

Node 0 access Node 1 access




Storage array

Volume

Disk group

VolumeVolume

A B

The disk groupis owned byNode 0

Node 0 access Node 1 access



Volume Manager Status

• Command line status is quicker

• Information can be more detailed

• Can be used in script files and cron jobs

• Detailed volume status using vxprint

• vxprint can also create a configuration file

• Disk status using vxdisk list

• Can provide critical physical path information



Volume Manager Status

# vxprint

Disk group: sdg0

TY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg sdg0 sdg0 - - - -

dm disk0 c4t0d0s2 - 8368512 - -dm disk7 c5t0d0s2 - 8368512 - -

v vol0 fsgen ENABLED 524288 - ACTIVEpl vol0-01 vol0 DISABLED 525141 - IOFAILsd disk0-01 vol0-01 ENABLED 525141 0 -pl vol0-02 vol0 ENABLED 525141 - ACTIVEsd disk7-01 vol0-02 ENABLED 525141 0 -pl vol0-03 vol0 DISABLED LOGONLY - IOFAILsd disk0-02 vol0-03 ENABLED 5 LOG -

# vxdisk list

DEVICE TYPE DISK GROUP STATUSc0t0d0s2 sliced - - errorc0t1d0s2 sliced disk02 rootdg online- - disk01 rootdg failed was:c0t0d0s2



Optimizing Recovery Times

Mirror synchronization time and file system recovery timescan be greatly reduced using the following techniques:

• Dirty region logging (DRL)

• Greatly reduces mirror synchronization time

• Veritas VxFS file system software

• Eliminates full file system checks by fsck



CVM and SSVM Post-Installation

• Initialize the rootdg disk group

• Three ways to satisfy this requirement

• Match the vxio major numbers between nodes

• The number must be the same on all nodes

• The number must not be duplicated on a single node

• Disable the DMP feature

• Enabled by default

• Should be completely removed



CVM and SSVM Post-Installation

C1

C2

Host system

DMPdriver

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Controller

Controller

Storage Array

SOCcard

SOCcard

fiber-opticinterface



Check Your Progress

• For the CVM and SSVM volume managers:

• Explain the disk space management techniques

• Describe the initialization process

• Describe grouping disk drives together

• List the basic status commands

• Describe the basic software installation process

• List the major post-installation issues

• Install and configure either CVM or SSVM



Think Beyond

Where does Volume Manager recovery fit into the highavailability environment?

What planning issues are required for the Volume Manager inthe high availability environment?

Is the use of the Volume Manager required for highavailability functionality?



Module 8

Cluster Configuration Database



Module Overview

• Objectives

• Relevance




• CDB database:

• Simple variable=value format

• Static content

• General cluster information

• CCD database:

• Database format

• Dynamic content

• Logical host configuration and status



Cluster Database Consistency

• Data propagation

• Database consistency checking

• Database majority



Cluster Database Consistency

Node 0

ccdd

CCD

(Master)

Node 1

ccdd

CCD

Node 2

ccdd

CCD

Updaterequest

SCI switch

FreezeFreeze

Propagate Propagate



Shared CCD Volume

• Two-node cluster only

• CCD on shared disk storage

• Enables CCD updates with only one node active

• Requires two dedicated shared disk drives



Shared CCD Volume

Node 1Node 0

Mass storage

I/O interfaces I/O interfaces

Mass storage

CCD primary CCD mirror

ccd.database ccd.database



Shared CCD Volume

# confccdssa clusternameThe disk group sc_dg does not exist.Will continue with the sc_dg setup.

Please, select the disks you want to use from the followinglist:

1) SSA:00000078C8A02) SSA:000000722F83Device 1: 11) t0d02) t0d13) t0d2Disk: 3

Disk c0t0d2s2 with serial id 00142458in SSA 00000078C8A0 has been selected as device 1.

Select devices from list.

1) SSA:00000078C8A02) SSA:000000722F83Device 2: 21) t0d2Disk:1) t0d2Disk: 1

Disk c2t0d2s2 with serial id 01186928in SSA 000000722F83 has been selected as device 2.

newfs: construct a new file system /dev/vx/rdsk/sc_dg/ccdvol:(y/n)? y



CCD Administration

The ccdadm command is used for CCD maintenance and isused to:

• Verify global CCD consistency

• Checkpoint the CCD

• Restore a CCD from a backup copy

• Disable the CCD quorum feature

• Identify CCD errors



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Describe the Cluster Database and its operation

• Describe the Cluster Configuration Database and itsoperation

• List the advantages of a shared CCD volume

• Manage the contents of the cluster configuration files



Think Beyond

When would you disable the CCD update quorumrequirement?

What would it take to have information defined for all nodes,even if the nodes are offline?



Module 9




Module Overview

• Objectives

• Relevance



PNM Overview

• Adapter failover

• IP address failover

• Continuous fault monitoring

• Backup group configuration

• Interface support



PNM Overview

hme0

Network

Node 0

ccd

NAFO groupIP address

Node 1

hme1

pnmd

Primary Backup

nafo7 group

Monitorprimary

/etc/pnmconfig

backup

NAFO groupconfiguration

hme0 hme1

Primary Backup

nafo12 group

ifconfig



The Network Monitoring Process

• What is wrong?

• Local adapter

• Remote adapter

• Network

• Take or request appropriate recovery action



How PNM Works

• PNM daemon is based on RPC model

• Configuration information is stored in the CCD

• PNM daemon accesses remote status over both publicand private networks

• Adapters supported with different Ethernet or FDDIMAC addresses on the same subnet



PNM Monitoring Routines

• TEST – Observes and solicits network activity

• DETERMINE_NET_FAILURE – Asks other clusternodes for status information

• FAILOVER – Creates a failover to the next adapter inthe group



The pnmset Command

• Configures network adapter backup groups

• Checks adapter status

• Picks primary adapter

• Activates only one adapter per group

• Saves the configuration

• Makes PNM daemon use new configuration

• Creates hostname. xxx file for only one adapter pergroup



The pnmset Command

# pnmset

In the following, you will be prompted to do configuration for network adapter failover

do you want to continue ... [y/n]: y

How many NAFO backup groups on the host [1]: 1

Enter backup group number [0]: 113

Please enter all network adapters under nafo113hme0 hme1 hme2

The following test will evaluate the correctnessof the customer NAFO configuration...

name duplication test passed

Check nafo113... < 20 seconds

hme1 is active

remote address = 192.9.10.222

nafo113 test passed



Other PNM Commands

• pnmstat – Queries backup group status

• pnmptor – Queries which adapter is active

• pnmrtop – Determines which backup group contains theadapter



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Explain the need for Public Network Management(PNM)

• Describe how PNM works

• Configure a NAFO group

• Disable the Solaris operating system Interface Groupsfeature



Think Beyond

Are there other system components that would benefit fromthe approach taken to network adapters by PNM?

What are the advantages and disadvantages of automaticadapter failover? Manual adapter failover?

How will IP striping affect this model? Can you realize thedual goals of higher throughput and high availability throughPNM/NAFO for the network connections?



Module 10

Logical Hosts



Module Overview

• Objectives

• Relevance



Logical Hosts

• Collection of network definitions and disk storage

• Highly available data services require a logical host

• Routines for logical host failover



Logical Host

Network

NAFO Group

129.50.20.3

Vol-02volume

# ping ds_host

# mount dshost:/Vol-02

Disk group: dg3

ccd.database

Node 0

Logical hostname:dshost

Primary: Node 0Backup: Node 1

Logical host name:lhost2

Node 1

ccd.database

lhost2

information

Data servicerecovery routines

Detect Node 0 failureImport dg3 disk groupfsck and mount Vol-02Ifconfig dshost IP addressOther recovery routines

Client workstation



Configuring a Logical Host

A logical host requires three main components:

• A NAFO backup group

• Created using the pnmset command

• A logical host definition in the CCD

• Created using the scconf -L command

• An administrative file system

• Created using the scconf -F command



Logical Host Variations

• Basic logical host

• Cascading failover

• Disabling automatic takeover

• Multiple disk groups and hostnames



Administrative File System Overview

• Required for each logical host

• Created by the scconf command

• Configured on special mirrored volume in logical hostdisk group

• Requires no management or modification

• Stores cluster configuration and logical host dataservice information



Creating the Administrative FileSystem

• Created with the scconf -F command option

• The command must be run on all cluster hosts

• Created on one of the logical host disk groups

• One for each logical host

• A special vfstab. logicalhost file is created

• Mount point for administrative file system



Logical Host File Systems

• Primary component of logical hosts

• User applications and data

• New file systems added manually

• Information placed in vfstab .lhost file

• Mount information used for logical host failover



Logical Host Control

• Forced logical host migration

• Use either haswitch or scadmin switch commands

• Initiate from current logical host master

• Logical host maintenance mode

• Use scadmin switch -m comand

• Typically for performing backups



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Configure logical hosts

• Create the administrative file system for a logical host

• Switch logical hosts between physical nodes



Think Beyond

If the concept of a logical host did not exist, what would thatimply for failover?

What complexities does having multiple backup hosts for asingle logical host add to the high availability environment?



Module 11

The HA-NFS Data Service



Module Overview

• Objectives

• Relevance



Objectives

• Describe the function of HA-NFS support files

• List the primary functions of HA-NFS start and stopmethods

• List the primary functions of HA-NFS fault monitoringprobes

• Configure HA-NFS in a Sun Cluster environment

• Add and remove HA-NFS file systems

• Switch a HA-NFS logical host between systems



HA-NFS Overview

• Completely transparent to NFS™ clients

• No client impact

• Supports NFS V.2 and V.3

• Co-exists with all other Sun Cluster data services

• Supports PC clients with lock recovery protocol

• PrestoServe, local access, Secure NFS, and Kerberosnot supported



HA-NFS Data Service

NFS

HA Framework

HA-NFS

User applications

Start methods

Stop methods

NFS-orientedfault monitoring



Start NFS Methods

• Starts automatically during reconfiguration

• Starts or restarts NFS-related daemons

• Forces NFS daemons to go through a lock recoveryprotocol

• Exports shared file systems

• Use only HA-NFS



Stop NFS Methods

• Runs during logical host reconfiguration

• Kills appropriate NFS-related daemons

• Unshares the file systems



HA-NFS Fault Monitoring

• Assesses health of HA-NFS data service

• Uses the public net to test the services

• Tests all HA-NFS shared file systems



Fault Probes

Service

Master node

(such as HTTP)

for logical host

Public net

Private netLocalprobe

Remoteeprobe

Backup nodefor logical host



Local Fault Probes

• Ensure NFS daemons are running on physical host

• Perform read, write, and locking operations

• Log failure messages and initiate giveaways



Remote Fault Probes

• Ensures the health of current logical host master

• Mounts and tests all HA-NFS file systems

• Tests read, write, and locking operations on each filesystem

• Can initiate logical host takeaway



Giveaway and Takeaway Process

• Local fault monitor initiates logical host giveaway

• Remote fault monitor initiates logical host takeaway

• Sanity check done before proceeding



Giveaway and Takeaway Process

phys-hostA

giveaway

Data service

check Localfaultmonitor

phys-hostB

Remotefaultmonitor

takeaway

check

Public network



Processes Related to NFS FaultMonitoring

• Logical host fault monitoring processes:

• nfs_probe_loghost

• nfs_mon

• nfs_probe_local_start



HA-NFS Support Files

To administer a HA-NFS logical host you must:

• Configure file system mount information

• Edit logical host-specific vfstab files

• Configure NFS share information

• Edit logical host-specific dfstab files



Using the hareg Command

Use the hareg command to:

• Register a standard data service

• Register a custom data service

• Unregister a data service

• Start and stop data services



File Locking Recovery

• HA framework exploits existing NFS locking recovery

• Server initiates recovery process

• Client supplies lock recovery information



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Describe the function of HA-NFS support files

• List the primary functions of HA-NFS start and stopmethods

• List the primary functions of HA-NFS fault monitoringprobes

• Configure HA-NFS in a Sun Cluster environment

• Add and remove HA-NFS file systems

• Switch a HA-NFS logical host between systems



Think Beyond

Are there restrictions on the file systems HA-NFS cansupport?

What types of NFS operations (if any) might be more difficultin the HA-NFS environment?



Module 12

System Recovery



Module Overview

• Objectives

• Relevance



Sun Cluster Reconfiguration Control

The following software components monitor for clusterfailures and can trigger a cluster reconfiguration:

• Cluster membership monitor

• Switch management agent

• Public network management

• Failfast (FF) driver

• Data service fault monitors

• Disk management software

• Database management software



Sun ClusterReconfiguration Control

Node 0 Node 1

Networkdriver

Networkdriver

Private networks

CMM

Heartbeats

DBMS

Diskmanagement

Fiber-opticchannels

Storage array

DBMS

Diskmanagement

Fiber-opticchannels

Storage array

PNMPNM

FFFF

Faultmonitor

Faultmonitor

CMM

SMASMA

ccdd ccdd

Updates



Sun Cluster Failfast Driver

• Memory resident driver (/dev/ff )

• Monitors critical daemons and operations

• Can force a UNIX panic if necessary

• Causes a cluster-wide reconfiguration



Sun Cluster Failfast Driver

All other nodes

Node 0

Critical daemonCritical operation

Failfast timeout

Reboot

Kernel driver: ff OK

Loss of heartbeat

reconf_ener

Clusterconfigurationdependentsteps

CMMdetected

UNIX panic



Sun Cluster Reconfiguration Sequence

• Controlled by a master script

• Can be initiated by operator commands

• Can be initiated by cluster monitoring software

• Both major and minor reconfigurations occur

• There are also independent reconfiguration processes



Sun ClusterReconfiguration Sequence

# scadmin startnode

# scadmin stopnode

Operator commands Status change detected

Failed private network

Other node failed

UNIX

Monitor and

disable structures

Resync volumes

File system recoveryReboot after panic

Virtual volumes

reconf_ener

Other node joining cluster

Varied reconfiguration steps

depending on the cluster

configuration and application

# scadmin startcluster

Disk Management



Sun Cluster Reconfiguration Steps

• Reconfiguration is globally coordinated

• Each step must complete on all host systems

• General reconfiguration has the highest priority

• Database recovery is performed second

• Data service recovery is performed last



Sun ClusterReconfiguration Steps

reconf_ener

Step 1

Step 2

Step 3

Step 4

Step n

|

|

|

|

reconf_ener

Step 1

Step 2

Step 3

Step 4

Step n

|

|

|

|

reconf_ener

Step 1

Step 2

Step 3

Step 4

Step n

|

|

|

|

CIS



Cluster Interconnect Failures

• The smad daemon detects CIS failures

• All nodes switch to their backup CIS interfaces

• The CIS failure is transparent to applications

• SCI interconnects require special repair steps



Two-Node Partitioned Cluster Failure

• Caused by a complete CIS failure

• Both nodes are isolated

• CVM and SSVM use basic quorum device

• SDS behavior is different



Logical Host Reconfiguration

• Local and remote fault monitors

• Both check the current logical host master

• Either can trigger a logical host reconfiguration

• Local fault monitor initiates a giveaway

• Remote fault monitor initiates a takeaway

• Sanity check must complete before reconfiguration



Logical Host Reconfiguration

phys-hostA

giveaway

Data service


phys-hostB

Remotefaultmonitor

takeaway

check

Public network



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• List the functions of Sun Cluster control software

• List the events that can trigger a cluster reconfiguration

• Explain the failfast concept

• Describe the general priorities during a clusterreconfiguration

• Describe the recovery process for selected clusterfailures

• Recover from selected cluster failures



Think Beyond

What are the issues for split-brain failures with more than twomodes?

Is it safe to have two “subclusters” running in a nominal four-node cluster?

What procedures should be documented for operationspersonnel?



Module 13

Sun ClusterHigh Availability Data Service API



Module Overview

• Objectives

• Relevance



Overview

Client-server

High Availability

Sun Cluster

User applications

data serviceC library

Command–line

utilitiesHigh Availability API

framework



Data Service Requirements

• Client-server data service

• Crash tolerant

• No dependencies on physical hostname of server

• Handles multi-homed hosts

• Handles additional IP addresses for logical hosts



Reconfiguration Overview

FM_STOP Method

STOP_NET Method

Configure logicalnetwork addresses DOWN

STOP Method

Stop/unmount volumes

Deport disk groups/disk sets

Give up logical host

FM_INIT

START_NET Method

Configure logicalnetwork addresses UP

START Method

Start/mount volumes

Import disk groups/disk sets

Take over logical host

FM_START Method

Logical Host Coming Down Logical Host Coming Up



Data Service Methods

• START

• STOP

• ABORT

• NET

• Fault Monitoring



Giveaway and Takeaway

phys-hostA

giveaway

Data service


phys-hostB

Remotefaultmonitor

takeaway

check

Public network



START and STOP Method Examples

Example 1

phys-mars phys-venus

mars venus

my_start (“mars”, “venus”, 30)

my_stop (“mars”, “venus”, 30)

my_start (“venus”, “mars”, 30)

my_stop (“venus”, “mars”, 30)



START and STOP Method Examples

Example 2

my_start (“”, “mars,venus”, 30)

my_stop (“”, “mars,venus”, 30)

my_start (“venus,mars”, “”, 30)

my_stop (“venus,mars”, “”, 30)

venus

mars

phys-mars phys-venus



Data Service Dependencies

A: START

A: START_NET

Configure logical networkaddresses up

B: START

B: START_NET

B: STOP

B: STOP_NET

Configure logical networkaddresses down

A: STOP_NET

A: STOP

START method ordering: STOP method ordering:

Data service A depends on Data service B



The haget Command

• Extracts Sun Cluster HA configuration information

• Called by data service methods

• Some haget command options need more information

• A logical hostname or a physical hostname

• A data service name



The hactl Command

• Provides control operations for fault monitors

• Request the movement of one or more logical hosts

• Request a cluster reconfiguration

• Performs sanity checks before proceeding

• Exits if sanity check fails



The halockrun Command

• Implements a mutex-type mechanism

• Serializes command execution



The hatimerun Command

• Sets a timeout for command execution

• Terminates command if not complete within time limit

• Runs in its own process group



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• Describe the available data service methods

• Describe when each method is called

• Describe how to retrieve cluster status information

• Describe how to retrieve cluster configurationinformation

• Describe how the fault methods work and how torequest failovers



Think Beyond

Are there other methods that might be needed for some dataservices? What would they be?

Are there ways to make a non-HA compliant data servicework with HA?

How would you debug HA API problems when you weredeveloping your data service?



Module 14

Highly Available DBMS



Module Overview

• Objectives

• Relevance



Sun Cluster HA-DBMS Overview

• Supports Oracle, Sybase, and Informix

• No changes to the database engine

• DBMS engine on local or multihost disks

• No change to database administration on clients

• Includes database fault monitors



Typical HA-DBMS ConfigurationNode a Node b

i1 i2 i3Private net

i1 – instance 1 (logical host X)i2 – instance 2 (logical host Y)i3 – instance 3 (logical host Z)



Configuring and Starting HA-DBMS

To prepare a HA-DBMS instance:

1. Configure the logical host using the scconfcommand.

2. Register the HA-DBMS service.

3. Start the HA-DBMS service using the haregcommand.

4. Register the HA-DBMS instances.

DBMS fault monitoring starts automatically when the HA-DBMS data service is started.



Stopping and UnconfiguringHA-DBMS

1. Stop the HA-DBMS service using hareg command.

2. Disconnect the service from the logical host using thescconf command.

3. Unregister the DBMS data service using the haregcommand.

4. Unconfigure the logical host, if appropriate, using thescconf command.

5. Remove the DBMS file systems from thevfstab. lhname file.



The HA-DBMS Start Methods

• Run automatically after hareg -y or the logical hoststarts

• Perform crash recovery if needed

• Always lets the HA framework start the database

• Start fault methods after this process completes



The HA-DBMS Stop and AbortMethods

• HA-DBMS Stop methods

• Run automatically, cleanly shuts down the database

• Allowed a 6 minute timeout to complete

• Logical host disk groups are deported

• HA-DBMS Abort methods

• Run if there is time during a node failure

• Immediate termination of database activity



HA-DBMS Fault Monitoring

• Local fault probes scan database alert files

• Consult the data service action file for alert message

• Performs the action defined for the error

• Remote fault probes act as a DBMS client

• Perform database queries

• Perform SQL table operations



Configuring HA-DBMS for HighAvailability

• Database volumes must be in switchable disk groups

• Archives might be an exception

• Database volume must be mirrored



Configuration Overview

• Never start the database manually

• Only the cluster framework should start the DBMS

• Database manager accounts on all nodes

• Be careful about default DBMS software locations

• Some installations will place on local private disks



Oracle Installation Preparation

• Start the cluster and HA-Oracle logical host

• Create dba accounts on primary and backup hosts

• Configure $ORACLE directories

• Install the Oracle software

• Change /etc/system on primary and backup hosts



Sybase Installation Preparation

• Start the cluster and HA-Sybase logical host


• Configure $SYBASE directories

• Install the Sybase software


• The ctlib.loc file must be loaded

• For use by the fault monitor software



Informix Installation Preparation

• Start the cluster and HA-Informix logical host


• Configure $INFORMIX directories

• Install the Informix software




Preparing the Logical Host

• Set database ownership of appropriate volume

• Set ownership with vxedit for CVM and SSVM

• Set ownership with chown/chgrp with SDS

• Grant fault monitors password access

• The process is different for each database

• Register the HA-DBMS data service

• Insert DBMS entry into the CCD

• Use haoracle /hasybase /hainformix insert



HA-DBMS Control

• Fault monitoring configuration information

• Stored in CCD

• Use haoracle /hasybase /hainformix insert

• Command variations for each HA-DBMS

• Fault monitoring control commands

• Use haoracle /hasybase /hainformix start/stop

• Does not start or stop the database



HA-DBMS Client Overview

• No special DBMS configuration

• Client must access only through logical hostname

• Never through the physical hostname

• Clients must be prepared for unscheduled disconnect

• As in a node crash



HA-DBMS Recovery

• Clients and applications must be prepared

• Transaction monitors can help with disruption

• Database instance recovery

• Volume manager recovery is part of it

• Might need to adjust restart delay time

• Set with haoracle /hasybase /hainformix insert

• Use haoracle /hasybase /hainformix update



HA-DBMS Configuration Files

• CCD insert information

• Oracle

• oratab , haoracle_config_V1

• Sybase

• sybtab , hasybase_config_V1

• Informix

• inftab , hainformix_config_V1



Exercise

• Objectives

• Tasks

• Discussion

• Solutions



Check Your Progress

• List the configuration issues for a highly availableDBMS instance

• Describe the general installation and configurationprocess for an HA-DBMS data service



Think Beyond

Are there services associated with HA-DBMS that should behighly available?

Are there advantages to using multiple disk groups with theHA-DBMS?

Do you need application-specific fault probes for an HA-DBMS environment? Why or why not?

Why is the quorum mechanism so important with HA-DBMS?

Copyright © 1999 Sun Microsystems, Inc., 901 San Antonio Road, Palo Alto, California 94303, U.S.A. All rights reserved.

This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of thisproduct or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Parts of the product may be derived from Berkeley BSD systems, licensed from the University of California. UNIX is a registered trademark in the U.S. and othercountries, exclusively licensed through X/Open Company, Ltd.

Sun, Sun Microsystems, the Sun Logo, Sun Enterprise, Sun StorEdge Volume Manager, Solstice DiskSuite, Solaris Operating Environment, Sun StorEdge A5000,Solstice SyMon, NFS, JumpStart, Sun VTS, OpenBoot, and AnswerBook are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and othercountries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Productsbearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering effortsof Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xeroxto the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s writtenlicense agreements.

U.S. Government approval required when exporting the product.

RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Govt is subject to restrictions of FAR 52.227-14(g) (2)(6/87) and FAR 52.227-19(6/87), or DFAR252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANYIMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THEEXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

ES-331 Sun Enterprise Cluster Administration Veritas Volume Manager

Documents

Transcript of ES-331 Sun Enterprise Cluster Administration Veritas Volume Manager