SPARC Enterprise - Fujitsu · Fujitsu attaches to the damage caused by not the one that the...

SPARC Enterprise - ETERNUS

SAN Boot Environment Build Guide

- Oracle Solaris -

Apr 2014

Version 1.7

Fujitsu Limited


Copyright 2007-2014 FUJITSU LIMITED

i

Use conditions The copyright, the trademark right, and other intellectual property rights

Contents (document, image, and voice, etc.) are protected by the copyright, the trademark right, and other

intellectual property rights.

The contents of this manual are printed out or can be downloaded within the range personally used. However,

our or the person entitled's permission is needed about use (recycling to your page and uploading to other

servers, etc.) excluding this.

Limitations of warranty

Fujitsu attaches to the damage caused by not the one that the accuracy and marketability and the adaptability

to purpose of use etc. are guaranteed but the use about the contents of this manual, and Fujitsu cannot

assume any juridical responsibility. The contents of this manual are subject to change and abolish without

notice.

Trademark Notice UNIX is a registered trademark of The Open Group in the United States and other countries.

All SPARC Enterprise, SPARC64, SPARC trademarks are used under license and are registered

trademarks of SPARC International, Inc. in the United States and/or other countries.

Oracle and Java are registered trademarks of Oracle and/or its affiliates.

All other trademarks or registered trademarks are the property of their respective companies.



ii

Preface

Content of this manual

This manual explains how to mount a 4Gbps Fibre Channel card (SE0X7F11x/SE0X7F12x)

on SPARC Enterprise and build a SAN Boot environment, in which the OS can be booted

from ETERNUS storage system.

Disk array devices other than ETERNUS are not described at this guide. See Chapter 2.1.1

Required Hardware.

Intended Audience

This manual is intended for builders and administrators of SAN Boot environments.

Organization

This manual consists of the following chapters:

Chapter 1 Overview

Introductory information about a SPARC Enterprise SAN Boot environment

Chapter 2 Hardware/Software Configuration

Patterns of configuration prerequisite to building a SAN Boot environment

Chapter 3 Precautions

Precautions in building and running a SAN Boot environment

Chapter 4 Building an OS Boot Environment

How to build a SAN Boot environment

Notes

The methods of notation are used in this manual:

Oracle Solaris 10 is noted as "Solaris 10."

Actual commands appear in boldface.

# /usr/sbin/FJSVpfca/fc_info -a <RETURN>



iii

Contents

CChhaapptteerr 11 OOvveerrvviieeww .......................................................................................................................................................................... 11

1.1 Configuration Patterns ................................................................................................5

1.1.1 Basic configuration ............................................................................................................................5

1.1.2 Disk array device mirroring configuration based on PRIMECLUSTER GDS ....................................7

1.1.3 Cluster configuration based on PRIMECLUSTER.............................................................................8

1.1.4 ETERNUS Configuration required to use the Advanced Copy function ............................................9

CChhaapptteerr 22 HHaarrddwwaarree//SSooffttwwaarree CCoonnffiigguurraattiioonn .............................................................................................. 1144

2.1 Hardware Environment .............................................................................................14

2.1.1 Required Hardware..........................................................................................................................14

2.1.2 Boot disk configuration ....................................................................................................................15

2.2 Software Environment ..............................................................................................17

2.2.1 Required software............................................................................................................................17

2.2.2 Optional software .............................................................................................................................19

CChhaapptteerr 33 PPrreeccaauuttiioonnss .................................................................................................................................................................. 2222

CChhaapptteerr 44 BBuuiillddiinngg aann OOSS BBoooott EEnnvviirroonnmmeenntt .............................................................................................. 2277

4.1 Creating a Boot Disk on a Disk Array Device............................................................29

4.1.1 Creating a boot disk using a network install server .........................................................................29

4.1.1.1 Creating a network install server ..................................................................................................30

4.1.1.2 Configuring a network install server..............................................................................................31

4.1.1.3 Labeling disks ...............................................................................................................................36

4.1.1.4 Configuring Custom JumpStart.....................................................................................................38

4.1.1.5 Configuring the Fibre Channel boot code.....................................................................................43

4.1.1.6 Executing network installation ......................................................................................................46

4.1.2 Creating a boot disk by copying an existing boot disk residing on an internal disk.........................47

4.1.2.1 Getting ready to copy the boot disk to a disk array device...........................................................48

4.1.2.2 Creating a boot disk......................................................................................................................48

4.1.2.3 Editing mount table information ....................................................................................................49

4.1.2.4 Configuring the Fibre Channel boot code.....................................................................................49

4.1.2.5 Resetting the server......................................................................................................................50

4.1.2.6 Booting from a disk array device ..................................................................................................51

4.2 Making the Path to a Boot Disk Redundant ..............................................................52

4.2.1 Applying the FUJITSU Recommended & Security Patches Cluster and PTF and installing the

Enhanced Support Facility..............................................................................................................52



iv

4.2.2 Configuring the ETERNUS Multipath Driver ....................................................................................53

4.2.2.1 Single system (non-cluster system)..............................................................................................53

4.2.2.2 Cluster system ..............................................................................................................................60

4.3 Boot Disk Mirroring ...................................................................................................70

4.3.1 Mirroring by PRIMECLUSTER GDS................................................................................................70

4.3.2 Notes on using PRIMECLUSTER....................................................................................................72

4.3.2.1 Cluster system building procedure ...............................................................................................72

4.3.2.2 Switching cluster system at the time of boot disk failures ............................................................72

AAppppeennddiixx AA BBoooott DDeevviiccee SSeettuupp CCoommmmaannddss .................................................................................................. 7733

A.1 Command Executable on the OS.............................................................................73

A.2 Command Executable on the OBP...........................................................................78

AAppppeennddiixx BB CChheecckkiinngg tthhee FFiibbrree CChhaannnneell CCaarrdd BBoooott CCooddee VVeerrssiioonn NNuummbbeerr

................................................................................................................................................................................................ 8877

B.1 Checking on the OS .................................................................................................87

B.2 Checking on the OBP...............................................................................................87

AAppppeennddiixx CC RReeccoorrddiinngg SSAANN BBoooott SSeettttiinngg IInnffoorrmmaattiioonn.................................................................... 8899

AAppppeennddiixx DD MMaakkiinngg FFiixxeess ttoo SSeettttiinngg FFiilleess aafftteerr aa BBoooott FFaaiilluurree.......................................... 9911

D.1 If the OS Has Been Installed As Per Section 4.1.1, "Creating a boot disk using a

network install server"..............................................................................................91

D.2 If the OS Has Been Installed As Per Section 4.1.2, "Creating a boot disk by copying

an existing boot disk residing on an internal disk" ...................................................92

AAppppeennddiixx EE FFiibbrree CChhaannnneell DDrriivveerr//BBoooott CCooddee AAuuttoo--TTaarrggeett BBiinnddiinngg FFuunnccttiioonnss......

................................................................................................................................................................................................ 9944

E.1 Fibre Channel Driver Auto-Target Binding Function .................................................94

E.2 Fibre Channel Boot Code Auto-Target Binding Function ..........................................96

AAppppeennddiixx FF SSAANN BBoooott rreelleeaassee pprroocceedduurree ........................................................................................................ 9977

F.1 ETERNUS Multipath Driver.......................................................................................97

AAppppeennddiixx GG UUppddaattiinngg tthhee FFiibbrree CChhaannnneell CCaarrdd BBoooott CCooddee .................................................. 110000

1

Chapter 1 Overview

The term SAN Boot refers to having an operating system (OS) or application stored in external SANstorage, not on an internal disk in a server, and starting (that is, booting) the OS or application from there.

This document describes the workflow for building a SAN Boot environment, in which a Fibre Channelcard is mounted on a server to start the OS from ETERNUS storage system (RAID).

Having an OS boot disk on an external disk array device offers the following advantages:

1. Enhanced availability

Use of a high-reliability disk array (RAID) device

Enhanced reliability results from managing the boot disk on a disk array (RAID) device.

Backup/restore work made more efficient

Use of the disk copy feature of a disk array device drastically cuts the period during whichbusiness is stopped for backing up and restoring system volumes. The CPU load incurred whilebacking up and restoring the system volumes is also reduced.

For more details, see Section 1.1.4, "ETERNUS Configuration required to use the Advanced Copyfunction."

2

NoteETERNUS SF AdvancedCopy Manager (ACM) or PRIMECLUSTER GDS Snapshot is requiredto use the disk copy feature (Advanced Copy feature) of ETERNUS (disk array device).

2. Greater ease of operation management

System volumes kept under consolidated management

Boot disks that were previously spread among multiple servers can be kept under consolidatedmanagement, as they are contained in a single disk array device.

Development environment generation management

Multiple development environments maintained on a single disk array device can be switched, asrequired. This eliminates the need to keep a server for each development environment, therebyallowing the number of servers and operational workload to be reduced.

3

3. Better maintainability

Handling of disk failures made simpler

If a disk (system volume) fails, the system administrator notifies a service engineer in charge ofthe fact and lets the engineer replace the disk for the system to recover automatically. The systemadministrator's workload is thus lightened.

Applying patches made simpler

Use of the disk copy feature of a disk array device trims the period of business stop for backing upsystem volumes before patches are applied on them. With the OS configured to boot from abackup volume (*1), if a problem occurs after applying the patches, the system can be rolled backto its status in effect prior to applying the patch by rebooting the server and switching the bootvolume. For more details, see Section 1.1.4, "ETERNUS Configuration required to use theAdvanced Copy feature."

(*1) PRIMECLUSTER GDS Snapshot provides this functionality with a simple commandoperation.

5

1.1 Configuration PatternsBoot the OS from an external disk array (RAID) device using Fibre Channel Cards in the patterns of theFibre Channel connection configuration shown below. Points to watch concerning each configurationpattern are also given.

1.1.1 Basic configuration

1. Using a disk array device from a single server

Implement a multipath configuration pattern based on the ETERNUS multipath disk driver,in which at least two routes of Fibre Channel connections are maintained between theserver and each disk array device.

If disk swaps occur, they could result in degraded application disk access performance.Actions that help prevent disk swaps include adding to the memory mounted on the serveror reducing the rate of memory usage by applications.

If a server that is not equipped with an internal disk is used, an install server is required toinstall the OS and recover the boot disk.

6

2. Using a disk array device from multiple servers

Fabric connection

FC-AL direct connection

Implement a multipath configuration pattern based on the ETERNUS multipath disk driver,in which at least two routes of Fibre Channel connections are maintained between eachserver and the disk array device.

If disk swaps occur, they could result in degraded application disk access performance.Actions that help prevent disk swaps include adding to the memory mounted on each serveror reducing the rate of memory usage by applications.

7

When a server panics, the other servers having their boot disks placed in the same RAIDgroup as the server might suffer degraded boot disk access performance for several tens ofseconds. See "2.1.2 Boot disk configuration."

If a server that is not equipped with an internal disk is used, a dedicated install server forinstalling the OS and recovering the boot disk is required.

1.1.2 Disk array device mirroring configuration basedon PRIMECLUSTER GDS

With the 1Gbps/2Gbps Fibre Channel card (PW008FC3), one card was required per path.With the single-channel 4Gbps Fibre Channel card (SE0X7F11x) and the dual-channel4Gbps Fibre Channel card (SE0X7F12x), in contrast, two Fibre Channel cards can becombined to build a disk array device mirroring configuration based on PRIMECLUSTERGDS, because multiple disk array device configurations in which boot disk is recognizedare enabled.

Implement a multipath configuration pattern based on the ETERNUS multipath disk driver,in which at least two routes of Fibre Channel connections are maintained between eachserver and a disk array device.

If disk swaps occur, they could result in degraded application disk access performance.Actions that help prevent disk swaps include adding to the memory mounted on each serveror reducing the rate of memory usage by applications.

When a server panics, the other servers having their boot disk placed in the same RAIDgroup as the server may suffer degraded boot disk access performance for several tens ofseconds.

8

If a server that is not equipped with an internal disk is used, a dedicated install server forinstalling the OS and recovering the boot disk is required separately.

1.1.3 Cluster configuration based on PRIMECLUSTER

A cluster system based on a SAN Boot environment may also be built.

1. Cluster configuration based on a single disk array device

9

2. Cluster configuration based on multiple disk array devices

1.1.4 ETERNUS Configuration required to use theAdvanced Copy function

On a traditional system on which the OS is booted from an internal disk, business needs to be stopped fora long period of time while backup/restore is executed using tape devices.

10

If backup/restore is executed using the ETERNUS Advanced Copy function (OPC/EC) in a SAN Bootenvironment, business can be carried out without stopping business while disk device copying is inprogress, thus drastically cutting the duration of business stop.

Backup/restore using the Advanced Copy function is operable in two ways: one using ETERNUS SFAdvancedCopy Manager (ACM) and one using PRIMECLUSTER GDS Snapshot (GDS Snapshot).

Using ETERNUS SF AdvancedCopy Manager (ACM)

Use of ACM offers these advantages:

Shorter period of business stop

Higher efficiency in backing up multiple servers (consolidated management)

Using PRIMECLUSTER GDS Snapshot (GDS Snapshot)

Use of GDS Snapshot offers the following advantages:

Shorter period of business stop

Restore made easier on a soft-mirroring configuration based on PRIMECLUSTERGDS

12

The table below contains a summary description of the features of ACM and those of GDS Snapshot.

Select the method suited to your system requirements.

NoteWhere system volumes are managed using PRIMECLUSTER GDS, their backup/restore is operable withboth ACM and GDS Snapshot, but use of GDS Snapshot is recommended for system volumes built on asoft-mirroring configuration.

ACM and GDS Snapshot features

: Advantage

ETERNUS SF AdvancedCopyManager (ACM)

PRIMECLUSTER GDS Snapshot

Operational server -

A dedicated server for executing thebackup/restore operations is required inaddition to the server to be backed upand restored.

No separate server is required, becausethe backup/restore operations areexecuted on the server to be backed upand restored.

Backup operation

Shut down the server to be backed uponce, and launch OPC from its backupserver. The server can be rebooted toresume suspended business withoutwaiting for physical copying tocomplete.

-

Reboot the server to be backed up insingle-user mode, and launch OPC.When physical copying completes,reboot the server in multi-user mode toresume suspended business.

Restore operation(mirroring byPRIMECLUSTERGDS notimplemented)

Shut down the server to be restoredonce, and launch OPC. The server canthen be rebooted to resume suspendedbusiness without waiting for physicalcopying to complete.

Suspended business can be resumed byrebooting the server to be restored andswitching the boot volume.

Restore operation(mirroring byPRIMECLUSTERGDS implemented)

-

The mirrored disk needs to bedisconnected and then reconnectedbefore and after the OPC restoreoperation.

-

When OPC physical copyingcompletes, switch back to the originalboot volume to resume suspendedbusiness.

Multi-serverbackup efficiency

Backup volumes on multiple serverscan be placed under consolidatedmanagement from a single backupserver.

-

The backup/restore operations areexecuted on each individual server.

Function forbooting from abackup volume

-

The mount point needs to be changedby editing the vfstab file.

The function can be easily configuredusing command.

13

For more feature details, refer to the ETERNUS SF AdvancedCopy Manager and PRIMECLUSTERGDS manuals.

Instructions on how to verify the progress of OPC physical copying are included in the ETERNUS SFAdvancedCopy Manager and PRIMECLUSTER GDS manuals.

14

Chapter 2 Hardware/SoftwareConfiguration

The hardware configuration and the software configuration described in this chapter are prerequisite tobooting the OS from an external disk array device using Fibre Channel cards.

2.1 Hardware Environment

2.1.1 Required Hardware

In this manual, the following hardware organization and the software configuration are supported.

Type Device name Remarks

Server SPARC Enterprise M3000/M4000/M5000/M8000/M9000/T1000/T2000/T5120/T5140/T5220/T5240/T5440

Fibre Channel cardmounted on servers

Single-channel 4Gbps Fibre Channel card [SE0X7F11x]

Dual-channel 4Gbps Fibre Channel card [SE0X7F12x]

Disk array device ETERNUS DX60 S2/DX80 S2/DX90 S2

ETERNUS DX400 S2 series

ETERNUS DX8000 S2 series

ETERNUS DX60/DX80/DX90

ETERNUS DX400 series

ETERNUS DX8000 series

ETERNUS2000 Model 50/100/200

ETERNUS3000 Model 80/100/300/500/700

ETERNUS4000 Model 80/100/300/400/500/600

ETERNUS6000 Model 500/700/900/1100

ETERNUS8000 Model 700/800/900/1100/1200

Fibre Channel switch ETERNUS SN200 Series

15

If a server that is not equipped with an internal disk is used, an install server is required separately toinstall and recover the OS.

2.1.2 Boot disk configuration

SAN Boot has a system disk placed on the ETERNUS disk array. In addition to system disks, a varietyof disk volumes are placed on the disk array device. The way these disk volumes are placed on the diskarray device could affect the performance of access to system disks residing on other servers and userdata disks such as databases. To keep disk access performance unaffected, take the following precautionin implementing the disk configuration:

Do not place in the same RAID group a system disk area and areas (system and data disks)that are accessible from other servers.

16

If any other kind of disk configuration is used, the following problem may occur:

If multiple system disks are placed in the same RAID group, the occurrence of disk swapscould result in degraded disk access performance for volumes residing in the same RAIDgroup.

17

Placing a system disk and a shared data area in the same RAID group would degradeaccess performance for the shared data area by several tens of seconds during a memorydump access session triggered by a server panic.

2.2 Software EnvironmentThis section describes the required software.

2.2.1 Required software

The following software components are required:

Software Version Remarks

Oracle Solaris Solaris 10 Solaris 10 11/06 or newer

It is necessary to installSolaris 10 8/07 or newerwhen SPARC Enterprise

T5120/T5140/T5220 is used.

It is necessary to installSolaris 10 5/08 or newerwhen SPARC EnterpriseT5240/T5440 is used.

It is necessary to install

18

Solaris 10 10/08 or newerwhen SPARC EnterpriseM3000 is used.

FUJITSU PCI Fibre Channeldriver

4.0 or newer Patch 914583-10 or newerneeds to be installed.

ETERNUS Multipath Driver 2.0.1 or newer Patch 914267-04 or newerneeds to be installed.

When using ETERNUS2000, use 914267-05 ornewer.

When using ETERNUSDX60/DX80, use 914267-09or newer.

When using ETERNUSDX90/DX4xx/DX8xxx, use914267-11 or newer.

If using ETERNUS DX60S2/DX80 S2/Dx90 S2, 3.0.3or newer, or 3.0.2 orolder+patch 914267-13 isnecessary.

SPARC EnterpriseT5140/T5240/T5440 requires3.0.3 or newer, or 3.0.2 orolder+patch 914267-13.

On Solaris 10 10/08 ornewer, 3.0.0 or newer, or2.0X+patch 914267-07 isnecessary.

If the kernel update patch144500-19 or newer isapplied to Solaris 10 9/10 orolder, 3.0.4 or newer, or3.0.3+patch 914267-14 isnecessary.

3.0.4 or newer is necessaryfor Solaris 10 8/11 or newer.

19

2.2.2 Optional software

PRIMECLUSTER GDS is required to implement system volume mirroring between disk arraydevices.


PRIMECLUSTER GDS 4.2 ornewer

It is necessary to apply the following patches when4.2A00 is used.

- Patch 914423-03 or newer needs to beinstalled.

- When using ETERNUS 2000, use 914423-05or newer.

- When using ETERNUSDX60/DX80/DX4xx/DX8xxx, use 914423-10or newer.

- For ETERNUS DX90, use 914423-17 ornewer.

- When create an alternate boot environment,use 914423-08 or newer.

When system volume is deployed on ETERNUSDX60/DX80/DX4xx/DX8xxx and a snapshot ofthe system volume or an alternative bootenvironment is created using PRIMECLUSTERGDS Snapshot and the advanced copy feature ofETERNUS, the following patches are necessary.

- For 4.2A00, 914423-26 or newer

- For 4.3A10, T006619SP-01 or newer

One of the following PRIMECLUSTER products is required to build a cluster system:


PRIMECLUSTER Enterprise Edition

PRIMECLUSTER HA Server

PRIMECLUSTER Lite Pack

4.2 ornewer

This product is bundled with PRIMECLUSTERGDS.

It is necessary to apply the following patcheswhen 4.2A00 is used.

- Patches 901201-06 or newer, 914325-03 ornewer, 914468-01 or newer and 914499-01or newer need to be installed.

20

ETERNUS SF AdvancedCopy Manager is required to execute backup/restore using ETERNUSAdvanced Copy function or PRIMECLUSTER GDS Snapshot function.


ETERNUS SF AdvancedCopyManager

13.0 ornewer

Both agents and the manager are Solaris 10-ready.

ETERNUS SF AdvancedCopyManager tape agent license

13.0 ornewer

Required to back up to tape with ETERNUS SFAdvancedCopy Manager.

One copy is required for each tape agent installed.

Tape agents are Solaris 10-ready.

ETERNUS SF AdvancedCopy Manager tapeserver option is required to be installed on tapeserver.

For more details on ETERNUS SF AdvancedCopy Manager, refer to the ETERNUS SFAdvancedCopy Manager manual.

PRIMECLUSTER GDS Snapshot is required to create snapshots of a system volume, or createan alternate boot environment using the ETERNUS Advanced Copy function or thePRIMECLUSTER GDS copy function.


PRIMECLUSTER GDS Snapshot 4.2 ornewer

A PRIMECLUSTER product bundled withPRIMECLUSTER GDS is required to use thisproduct.

It is necessary to apply the following patches when4.2A00 is used.

- Patch 914457-02 or newer needs to beinstalled.

- When using ETERNUSDX60/DX80/DX90/DX4xx/DX8xxx, use914457-03 or newer.

- When using Solaris 10 10/08 or newer, use914457-03 or newer.

- When using the patch (137137-09 or newer)that corresponds Solaris 10 10/08 or newer,use 914457-03 or newer.

21

For more details on PRIMECLUSTER GDS Snapshot, refer to "PRIMECLUSTER Global DiskService Guide."

22

Chapter 3 Precautions

1. The single-channel 4Gbps Fibre Channel card (SE0X7F11x) and dual-channel 4Gbps Fibre Channelcard (SE0X7F12x) support a boot code to allow the OS to be booted from a disk array deviceconnected to the Fibre Channel card. The Fibre Channel cards come with this boot code disabled bydefault. The boot code needs to be enabled before the OS can be booted using a Fibre channel card.Enabling or disabling the boot function on either port of the dual-channel 4Gbps Fibre Channel card(SE0X7F12x) will automatically impart the same setting to the other port. It is not possible to make achange to the setting of either port alone and not both.For information about enabling and disabling the boot function on Fibre Channel cards, see AppendixA.2 1 fjpfca-set-bootfunction.

2. Be sure to install the OS on the boot disk using one of the methods described in Chapter 4.It is not possible to copy any boot disk that has been used on another host by using the dd commandor using ETERNUS EC (Equivalent Copy) or OPC (One Point Copy) and use it.Boot disks cannot be created in any procedure other than that described in Chapter 4.

3. Keep a record of the boot configuration information set on the Fibre Channel card.The boot configuration information is required when the Fibre Channel card is replaced, because itneeds to be reproduced on the new Fibre Channel card.For information on the kinds of information to be recorded, see Appendix C, "Recording SAN BootSetting Information."

4. Fujitsu recommends not placing in the same RAID group the boot disks for different hosts.For more details, see Section 2.1.2, "Boot disk configuration."

23

5. Do not create a huge file or a large number of files in /tmp (tmpfs).If a huge file or a large number of files are created in /tmp (tmpfs), the system could slow down due toa lack of sufficient memory available to it.

6. When using PRIMECLUSTER or PRIMECLUSTER GDS to build a system implementation, seeSection 4.3.2, "Notes on using PRIMECLUSTER," for additional tips.

7. The access path settings based on WWN (World Wide Port Name) described below arerecommended for the ETERNUS disk array device and the ETERNUS SN200 Fibre Channel switch.These settings remove the need to reconfigure the following devices for using the resourcemanagement software, Systemwalker Resource Coordinator, facilitating the transition process andspeed up the transition.

ETERNUS disk array device

Configure as host table or enable the host affinity feature with regard to the ETERNUS FC-CAport, and set the WWN of the Fibre Channel card used as a host World Wide Name. Refer to eachServer Connection Guide for the setting of the ETERNUS disk array device.

ETERNUS SN200 Fibre Channel switch

Using the WWN of the Fibre Channel card and that of each disk array device, configure a one-to-one WWN zoning plan, which establishes zoning using the WWN of the host HBA port and thatof the FC-CA port.

Note) In this book, "Port Name" is replaced by "WWN".

8. Disks bearing the EFI (Extensible Firmware Interface) disk label cannot be used as the boot disk.The EFI disk label supports physical disk volume and virtual disk volume of 1Tbyte on Solaris 105/09 or older and 2Tbyte on Solaris 10 10/09 or newer. However, booting cannot be performed ondisks with EFI disk labels.

24

9. The warning message shown below is displayed when a multipath is created. This message may beignored.This message reports that the disk array device has received a SCSI RESET that is issued at multipathbuild time and does not relate to any disk array device or server operation.This message will be displayed only when a new multipath is built or additions are made to a fleet ofdisk array devices. If message monitoring is implemented, disable the monitoring process temporarilyor simply ignore the output message when it is displayed.

WARNING: /pci@1,700000/fibre-channel@0/sd@10,0 (sd805) :

Error for Command: write (10) Error Level: Retryable

Requested Block: 5651696 Error Block: 5651696

Vendor: FUJITSU Serial Number: 0000080115

Sense Key: Unit Attention

ASC: 0x29 (bus device reset message occurred) , ASCQ: 0x3, FRU: 0x1

10. If a system disk is mirrored using PRIMECLUSTER GDS, the following message may be displayedat boot time, but simply ignore it, because there is no actual problem with the system:

NOTICE: "forceload: drv/<driver name> appears more than once in /etc/system.

* One of mplb, mplbt and sd appears in place of <driver name>.

This message is displayed if the setting forceload is defined in duplicate in /etc/system file. Toleave this message hidden, delete later occurrences of the setting of forceload.

forceload: drv/mplb

~

forceload: drv/mplb Delete this line.

25

11. In configuring the Fibre Channel driver, the link speed (transmission line speed) setting can be set toautomatic selection to facilitate connectivity, but the expected link speed (especially for the 4Gbps)may not be attained depending on the connection timing. Enter the link speed setting in/kernel/drv/fjpfca.conf.

Example: Set fjpfca0 to a link speed of 4Gbps.

port=

"fjpfca0:nport:sp4";

For instructions how to set a link speed, refer to "FUJITSU PCI Fibre Channel Guide."

12. Be sure to configure LUN 0(Host Logical Unit Number 0) on ETERNUS. The LUN 0 is used torecognize ETERNUS by Fibre Channel boot code.This problem has been fixed in boot code V12L50. Update the boot code by referring to "Appendix GUpdating the Fibre Channel Card Boot Code".

13. When you install the Solaris 10 10/08 or newer by the installation server, the installation servershould work on Solaris 10 10/08 environment or newer or on Solaris 10 environment that 137137-09or newer patch are applied; otherwise driver packages cannot be installed.

14. In boot code V12L30 installed on Fibre Channel card (SE0X7F11F, SE0X7F12F), the OS bootingprocess might hang up for the following conditions.

When connecting it with 8G port of the SN200 series

When booting it after about 25 days or more pass from the power supply turning on of theserver device

This problem is corrected in boot code V12L40. Referring to "Appendix G Updating the FibreChannel Card Boot Code", update the boot code.When hanging up occurs, turn the power off, and then turn the power on, and boots again.

15. The composition of the system volume larger than 512Gbyte is not supported.This problem has been fixed in boot code V12L50. Update the boot code by referring to "Appendix GUpdating the Fibre Channel Card Boot Code".

16. When Fibre Channel Cards are replaced, update the boot code referring to "Appendix G: Updatingthe Fibre Channel Card Boot Code".

17. In a system with redundant boot devices, even if several boot devices are set, the system may not bebooted from another path, if it is supposed to, due to the actions taken at the time of fault detection.At the time of OS startup, if the startup cannot be done from the first registered boot device, attemptsare made to boot from the other registered boot devices in succession. However, if any problemoccurs after the OS has been started from any of these devices, attempt to boot from any other devicesis not performed.

18. A SAN boot configuration that uses the ZFS filesystem as the system volume, is not supported.

26

19. Panic may occur on a system with the combination of the ETERNUS multipath driver 3.0.3 or olderand Solaris 10 8/11 or the kernel update patch 144500-19 or newer.If using Solaris 10 8/11 or newer, or Solaris 10 9/10 or older, along with kernel update patch 144500-19, use ETERNUS multipath driver 3.0.4.If using Solaris 10 9/10 or older and ETERNUS multipath driver 3.0.3 or older, apply 914267-14before applying kernel update patches.

20. On SPARC Enterprise T5140/T5240/T5440 servers, when multipath is configured with ETERNUSmultipath driver or if it is removed, the system may fail to start.If SPARC Enterprise T5140/T5240/T5440 is used and Solaris 10 10/09 or older is installed andmultipath is configured but 914267-12 or newer is not applied, install 914267-12 or newer beforeapplying 142909-17 or newer or the corresponding PTF/FJ RSPC. (If 914267-12 had already beeninstalled, installing 142909-17 or newer or the corresponding PTF/FJ RSPC will prevent thissymptom from occurring).If SPARC Enterprise T5140/T5240/T5440 is used and Solaris 10 9/10 or older is installed or thekernel update patch 142909-17 or newer is applied to a Solaris 10, apply 914267-12 or newer andbefore rebooting the server on which this symptom occurs, normalize the links of internal disks withthe "disks" or "devfsadm" commands.

21. In a SAN boot environment, if the configuration of boot code is changed in the middle of I/Oprocessing, I/O may hang up.This problem has been fixed by 914583-10. Apply 914583-10 or newer.

27

Chapter 4 Building an OS BootEnvironment

This chapter describes the following topic:

Building an OS boot environment from a disk array device

Before performing this procedure, set zoning with the FC Switch (fabric connection), set security, set upa disk array device to make available a LUN in which to create a boot disk.

If a server that is not equipped with an internal disk is used, only the method described in Section 4.1.1,"Creating a boot disk using a network install server," can be used. Since the FUJITSU PCI Fibre Channeldriver is not contained in Oracle Solaris, OS boot environment cannot be built using CD/DVD of OracleSolaris.

Boot environments cannot be built in any procedure other than that described in this guide.

In Section 4.1.1, "Creating a boot disk using a network install server," the method of identifying the diskarray device to become a boot disk can be automated. This method facilitates the job of configuring theFibre Channel driver and the Fibre Channel boot code. For more details, see Section 4.1.1.2,"Configuring a network install server."

If a disk array device is identified through automatic setting, set zoning with the FC Switch to disable aFibre Channel card from connecting to multiple disk array devices. In an environment in which a FibreChannel card is allowed to connect to multiple disk array devices, implement disk array deviceidentification manually (manual setting).

For information on building a cluster environment, see Section 4.3.2, "Notes on usingPRIMECLUSTER."

The workflow for building an OS boot environment is as follows:

28

Install the OS as instructed in Section 4.1, "Creating a Boot Disk on a Disk Array Device," and create aboot disk on a disk array device.

Then, boot the OS in single-path mode as instructed in Section 4.1.2.6, "Booting from a disk arraydevice."

Lastly, define a multipath configuration as per Section 4.2, "Making the Path to a Boot Disk Redundant."

29

4.1 Creating a Boot Disk on a Disk Array DeviceThere are two ways to create a boot disk on a disk array device, as follows:

1. Creating a boot disk using a network install server2. Creating a boot disk by copying an existing boot disk residing on an internal disk (only if the server isequipped with an internal disk)

4.1.1 Creating a boot disk using a network installserver

Executing a network install requires an install server, in addition to the host (install machine) that uses adisk array device as a boot device.

Work with the install machine from the install machine console. Work with the install server from aterminal, which is marked as "(INSTALL SERVER)" in the examples appearing in this guide.

30

4.1.1.1 Creating a network install server

Configure an install server to execute a network install. For more details on the work of creating aninstall server, refer to "Solaris 10 Release and Installation Collection" at Oracle Documentation.

If there are multiple hosts that use a disk array device as a boot device each, an OS image needs to becreated for each host. The hosts may share a single install image, however, in these situations:

1. Hosts use AL direct Fibre Channel connection, sharing the same values of target ID and max throttle.2. Hosts use automatically configured disk array devices on a FC switch connection, sharing the samevalues of target ID and max throttle.

Note) Multiple hosts which are the same architecture type can share the same OS install image,but the same OS install image can not be used for the other architecture type. SPARC EnterpriseM3000/M4000/M5000/M8000/M9000 (which are sun4u) can share the same OS install image.SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440 (sun4v) can share the sameOS install image.

A sample flow for creating an image of Solaris10 is described below.

In this flow, /export/install/Solaris10_hostname is used as the name of a sample directory in which anOS image is created.

1. Become a superuser on the install server.

(INSTALL SERVER) % su - <RETURN>

Password: password

2. Make a directory in which to create an OS image.

(INSTALL SERVER) # mkdir /export/install <RETURN>

(INSTALL SERVER) # cd /export/install <RETURN>

(INSTALL SERVER) # mkdir Solaris10_hostname <RETURN>

Include the host name hostname of the install machine in the created directory name to allowmanagement by host.

3. Mount the Solaris 10 Operating System DVD-ROM.

(INSTALL SERVER) # cd /cdrom/cdrom0/s0/Solaris_10/Tools <RETURN>

(INSTALL SERVER) # ./setup_install_server /export/install/Solaris10_hostname<RETURN>

4. When copying of the Solaris 10 Operating System completes, demount the DVD-ROM.

(INSTALL SERVER) # cd / <RETURN>

(INSTALL SERVER) # eject cdrom <RETURN>

31

4.1.1.2 Configuring a network install server1. Register the IP/MAC address of an installation target machine on the network install server.

○ Register the IP address of an installation target machine

Edit the /etc/hosts file with a text editor.

When an IP address is "192.168.1.1", it becomes the following.

192.168.1.1 hostname

○Register the MAC address of an installation target machine

Edit the /etc/ethers file with a text editor.

When an IP address is "0:80:17:28:1:f8", it becomes the following.

0:80:17:28:1:f8 hostname

2. For boot the installation target machine from the network, execute add_install_client command on thenetwork install server.

An add_install_client parameter changes with machine models.

SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440

(INSTALL SERVER) # cd /export/install/Solaris10_hostname/Solaris_10/Tools/ <RETURN>(INSTALL SERVER) # ./add_install_client hostname sun4v <RETURN>

SPARC Enterprise M3000/M4000/M5000/M8000/M9000

(INSTALL SERVER) # cd /export/install/Solaris10_hostname/Solaris_10/Tools/ <RETURN>(INSTALL SERVER) # ./add_install_client hostname sun4u <RETURN>

3. Install the Fibre Channel driver in the OS install image on the network install server.

This step makes the disk array device connected to the Fibre Channel card identifiable. Mount"FUJITSU PCI Fibre Channel 4.0" on the CD-ROM drive in the network install server and do thefollowing:

The process of installing the Fibre Channel driver varies with each target machine model.

Solaris 10 5/08 or older


(INSTALL SERVER) # cd /cdrom/cdrom0 <RETURN>

(INSTALL SERVER) # bin/pfcapkgadd.sh -R/export/install/Solaris10_hostname/Tools/Boot/ -p sun4v <RETURN>



(INSTALL SERVER) # bin/pfcapkgadd.sh -R/export/install/Solaris10_hostname/Tools/Boot/ -p sun4u <RETURN>

Solaris 10 10/08 or newer

(1) Create the working directory for unpacking miniroot.

32

(INSTALL SERVER) # mkdir /tmp/work <RETURN>

(2) Unpack the miniroot to the work directory using the root_archive(1M) command.

If /tmp/work/tmp/AdDrEm.lck file is not exist, following procedure ignoring.

(INSTALL SERVER) # /boot/solaris/bin/root_archive unpackmedia/export/install/Solaris10_hostname /tmp/work <RETURN>

(INSTALL SERVER) # rm /tmp/work/tmp/AdDrEm.lck <RETURN>

- The following messages might be displayed when root_archive command is executed. Youmight ignore these messages.

umount: /tmp/mnt29984 busy

rmdir: directory "/tmp/mnt29984": Directory is a mount point or in use

lofiadm: could not unmap file /export/install/Solaris10_hostname/boot/sparc.miniroot:Device busy


(3) Install the Fibre Channel driver in the working directory.



(INSTALL SERVER) # bin/pfcapkgadd.sh -R /tmp/work/ -p sun4v <RETURN>



(INSTALL SERVER) # bin/pfcapkgadd.sh -R /tmp/work/ -p sun4u <RETURN>

(4) Pack the working directory.

(INSTALL SERVER) # mkdir -p /tmp/media/Solaris_10 <RETURN>

(INSTALL SERVER) # /tmp/work/boot/solaris/bin/root_archive packmedia /tmp/media/tmp/work <RETURN>

The error messages might be displayed when root_archive command is executed. You mightignore these messages.

(5) Copy the file in /tmp/media directory to the installation image on the install server.

Target device path name for "umount -f" and "lofiadm -d" commands can confirm by "df -k"command. If /dev/lofi/X does not exist, proceed to the next step.

33

(INSTALL SERVER) # cd /tmp/media <RETURN>

(INSTALL SERVER) # find boot Solaris_10/Tools/Boot | cpio -pdum/export/install/Solaris10_hostname <RETURN>

(INSTALL SERVER) # umount -f /dev/lofi/1 <RETURN>

(INSTALL SERVER) # lofiadm -d /dev/lofi/1 <RETURN>


4. Verify the correspondence between the Fibre Channel card location and driver instance number.

Boot the installation target machine from the network in single-user mode, with the -s optionspecified.

ok boot net -s <RETURN>

5. Execute the following to verify the correspondence between the Fibre Channel card device path andthe driver instance:

# grep fjpfca /tmp/root/etc/path_to_inst <RETURN>

"/pci@1,700000/fibre-channel@0" 0 "fjpfca"

"/pci@2,600000/fibre-channel@0" 1 "fjpfca"

Each line of this command listing contains device path, instance number and instance name in thisorder.

In the example above, the drive instance of the Fibre Channel card mounted at device path"/pci@1,700000/XXXX@0" is fjpfca0.

The correspondence between device paths and server slot positions is described in the relevantserver’s user's guide. If any other server is used or if the relevant user's guide is not available forreference, check the correspondence in the following way:

6. Flash the LED on the Fibre Channel card associated with the driver instance. The flashing LED letsyou identify the Fibre Channel card location for a single-channel 4Gbps Fibre Channel card(SE0X7F11x), or the Fibre Channel card location and the port position associated with the driverinstance for a dual-channel 4Gbps Fibre Channel card (SE0X7F12x). The LED at fjpfca0 can beflashed as instructed below. The LINK LED will flash for 3 minutes.

# /usr/sbin/FJSVpfca/fc_adm -l fjpfca0 <RETURN>

To stop the flashing of the LED, enter Ctrl-c (press the c key while holding down Ctrl key).

For information on using fc_adm, refer to "FUJITSU PCI Fibre Channel Guide."

7. Shutdown the OS on the installation target machine and return to OBP.

34

# /usr/sbin/shutdown -i0 -g0 -y <RETURN>

8. Configure the definition files that are used for booting the OS at network install time.When Solaris 10 10/08 or newer is installed, the following procedures are executed beforehand.


Unpack the miniroot to the work directory using the root_archive(1M) command.


- The following messages might be displayed when root_archive command is executed. Youmight ignore these messages.





The files require the following setting on the network install server:

Fibre Channel driver setting file


{OS install path on the install server}/Tools/Boot/kernel/drv/fjpfca.conf


/tmp/work/kernel/drv/fjpfca.conf

Configure the Fibre Channel driver setting file (fjpfca.conf) to make the disk arraydevice on which to create a boot disk identifiable to the Fibre Channel driver. On adirect connection (FC-AL), the specification of port, fcp-auto-bind-function and fcp-bind-target is not mandatory. For more information on configuring fjpfca.conf, referto "FUJITSU PCI Fibre Channel Guide."

There are two ways to make a disk array device identifiable on a fabric connection(using the FC Switch): automatic setting and manual setting. Automatic setting formaking a disk array device identifiable, when selected, offers the following benefits:

1. If multiple hosts use a disk array device as a boot device each, they can stillshare and use the Solaris OS install image on the install server.

2. The Fibre Channel driver is made easier to configure.

These two methods of identifying a disk array device are described below.

a. [Automatic setting]Example: Set a fabric connection as a topology, a link speed of 4Gbps andautomatic setting as method of disk array device identification for fjpfca0.

35

port="fjpfca0:nport:sp4";

fcp-auto-bind-function=1;

port Define a connection topology type and a link speed.

fcp-auto-bind-function Configure a disk array device to be identifiedautomatically.

For more information about the automatic identification function, see AppendixE, "Fibre Channel Driver/Boot Code Auto-Target Binding Functions."

If a disk array device is identified through automatic setting, set zoning withthe FC Switch to disable a Fibre Channel card from connecting to multiple diskarray devices. In an environment in which a Fibre Channel card is allowed toconnect to multiple disk array devices, allow disk array devices to be identifiedthrough manual setting.

For information on how to set zoning with the FC Switch, refer to the relevantFC switch’s manual.

b. [Manual setting]Example: Set a fabric connection as a FC switch topology, a link speed of4Gbps and binding of a disk array device with target ID 16 for fjpfca0.

port="fjpfca0:nport:sp4";

fcp-bind-target="fjpfca0t16:0x210000c0004101d9";

port Define a connection topology type and a link speed.

fcp-bind-target Specify the target WWN.

Target driver setting file


{OS install path on the install server}/Tools/Boot/kernel/drv/sd.conf


/tmp/work/kernel/drv/sd.conf

Configure the target driver setting file (sd.conf) to make the logical unit (LUN)of the disk array device on which to create a boot disk identifiable. Define onlythe boot disk on the disk array device. If an identifiable logical unit of the diskarray unit is already defined, the entry may be bypassed.

Example: Identify target ID 16 and logical unit 0.

name="sd" class="scsi" target=16 lun=0;

When Solaris 10 10/08 or newer is installed, the following procedures areexecuted beforehand.

(1) Pack the working directory.


36

(INSTALL SERVER) # /tmp/work/boot/solaris/bin/root_archive packmedia/tmp/media /tmp/work <RETURN>

The error messages might be displayed when root_archive command is executed.You might ignore these messages.

(2) Copy the file in /tmp/media directory to the installation image on the install server.

Target device path name for "umount -f" and "lofiadm -d" commands can confirm by"df -k" command. If /dev/lofi/X does not exist, proceed to the next step.





The error messages might be displayed when root_archive command is executed.You might ignore these messages.

4.1.1.3 Labeling disks1. Boot the installation target machine from the network in single user mode


2. Create a disk label in the LUN that is used as a boot disk by carrying out the format (1M) command,and then check the size of the LUN.

# format <RETURN>

Searching for disks...done

AVAILABLE DISK SELECTIONS:0. c7t16d0 <FUJITSU-ETERNUS-4000 cyl 1038 alt 2 hd 64 sec 256>

/pci@1,700000/fibre-channel@0/sd@10,0Specify disk (enter its number): 0<RETURN>selecting c7t16d0[disk formatted]Disk not labeled. Label it now? y <RETURN>FORMAT MENU:

disk - select a disktype - select (define) a disk typepartition - select (define) a partition tablecurrent - describe the current diskformat - format and analyze the diskrepair - repair a defective sectorlabel - write label to the diskanalyze - surface analysis

37

defect - defect list managementbackup - search for backup labelsverify - read and display labelssave - save new disk/partition definitionsinquiry - show vendor, product and revisionvolname - set 8-character volume name!<cmd> - execute <cmd>, then returnquit

format> partition <RETURN>

PARTITION MENU:0 - change '0' partition1 - change '1' partition2 - change '2' partition3 - change '3' partition4 - change '4' partition5 - change '5' partition6 - change '6' partition7 - change '7' partition

select - select a predefined tablemodify - modify a predefined partition tablename- name the current tableprint - display the current tablelabel - write partition map and label to the disk!<cmd> - execute <cmd>, then returnquit

partition> print <RETURN>Current partition table (original):Total disk cylinders available: 4254 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks0 root wm 0 – 15 128.00MB (16/0/0) 2621441 swap wu 16 – 31 128.00MB (16/0/0) 2621442 backup wu 0 – 4253 33.23GB (4254/0/0) 696975363unassigned wm 0 0 (0/0/0) 04unassigned wm 0 0 (0/0/0) 05unassigned wm 0 0 (0/0/0) 06 usr wm 32 – 4253 32.98GB (4222/0/0) 691732487unassigned wm 0 0 (0/0/0) 0

format> quit <RETURN>

3. Shutdown the OS on the installation target machine and return to OBP.


38

4.1.1.4 Configuring Custom JumpStart

Configure Solaris custom jump start to install a driver package. If Custom JumpStart is used, a driverpackage is automatically installed and configured at the same time as Solaris is installed.

Perform this procedure on the install server.

1. Make a Custom JumpStart directory.

Make and share a jumpstart directory on the install server.

(INSTALL SERVER) # mkdir /jumpstart <RETURN>

(INSTALL SERVER) # share -F nfs -o ro,anon=0 /jumpstart <RETURN>

2. Copy a driver package, patch and install file.Copy a driver package, patch and install file to the jumpstart directory on the install server.

Copy the CD image of FUJITSU PCI Fibre Channel to the jumpstart directory on the installserver.

(INSTALL SERVER) # mkdir /jumpstart/FJPFCA <RETURN>


(INSTALL SERVER) # find . | cpio -pumd /jumpstart/FJPFCA <RETURN>

When FUJITSU PCI GigabitEthernet/FUJITSU ULTRA LVD SCSI Host Bus Adapter Driverinstall, the following execute.

FUJITSU PCI GigabitEthernet 3.0 Update1 or newer

Copy the CD image of FUJITSU PCI GigabitEthernet 3.0 Update1 or newer to thejumpstart directory on the install server.

(INSTALL SERVER) # mkdir /jumpstart/fjgi <RETURN>

(INSTALL SERVER) # cp -p /cdrom/cdrom0/install /jumpstart/fjgi/. <RETURN>

(INSTALL SERVER) # cp -p /cdrom/cdrom0/admin /jumpstart/fjgi/. <RETURN>

(INSTALL SERVER) # cp -pr /cdrom/cdrom0/FJSVgid_3.0/10/* /jumpstart/fjgi/.<RETURN>

FUJITSU PCI GigabitEthernet 4.0 or newer

Copy the CD image of FUJITSU PCI GigabitEthernet 4.0 or newer to the jumpstartdirectory on the install server.

(INSTALL SERVER) # mkdir /jumpstart/fjgi <RETURN>

39

(INSTALL SERVER) # cp -p /cdrom/cdrom0/install_v4 /jumpstart/fjgi/. <RETURN>

(INSTALL SERVER) # cp -p /cdrom/cdrom0/admin /jumpstart/fjgi/. <RETURN>

(INSTALL SERVER) # cp -pr /cdrom/cdrom0/FJSVgid_4.0/10/* /jumpstart/fjgi/.<RETURN>

FUJITSU ULTRA LVD SCSI Host Bus Adapter Driver

Copy the CD image of FUJITSU ULTRA LVD SCSI Host Bus Adapter Driver to thejumpstart directory on the install server.

(INSTALL SERVER) # mkdir /jumpstart/fjulsa <RETURN>

(INSTALL SERVER) # cp -p /cdrom/cdrom0/install /jumpstart/fjulsa/. <RETURN>

(INSTALL SERVER) # cp -p /cdrom/cdrom0/admin /jumpstart/fjulsa/. <RETURN>

(INSTALL SERVER) # cp -pr /cdrom/cdrom0/ultra_lvd_driver/10/*/jumpstart/fjulsa/. <RETURN>

3. Copy a JumpStart sample

Copy the JumpStart sample file from the OS install image.

(INSTALL SERVER) # cp -r/export/install/Solaris10_hostname/Solaris_10/Misc/jumpstart_sample/* /jumpstart<RETURN>

4. Edit the profile

Edit the /jumpstart/profile file with a text editor.

Create a profile to meet the installation target machine configuration. Create one as instructed in"Solaris Installation Guide: Custom JumpStart and Advanced Installations."

Profile sample setting :

install_type initial_install # The install_type parameter is required. Specify Initial_install.

system_type server # Specifies server as system_type.

partitioning explicit # Specifies explicit as partitioning.

cluster SUNWCXall # Specifies SUNWCXall (Entire Software Group Plus OEM Support)as an installed OS cluster.

filesys c7t16d0s1 4096 swap # Assigns a swap file system space of 4096 MB toc7t16d0s1.

filesys c7t16d0s0 free / # Assigns the rest of the disk space to c7t16d0s0.

5. Copy a finish script sample

40

Copy a sample of the finish script from the FJPFCA directory to the /jumpstart directory as finish.

(INSTALL SERVER) # cp/jumpstart/FJPFCA/FJPFCA4.0/tool/FJPFCA_jumpstart_finish.sample/jumpstart/finish <RETURN>

6. Edit the finish script

Edit /jumpstart/finish with a text editor. Edit the following parameters:

JUMPSTART_HOST Specifies the host name or IP address of the install server.

JUMPSTART_DIR Specifies the directory in which the JumpStart setting file is stored.Edit this parameter only when a directory other than /jumpstart is used.

When FUJITSU PCI GigabitEthernet/FUJITSU ULTRA LVD SCSI Host Bus Adapter Driverinstall, add the following contents. In the following examples, it adds it under"PF_ARCH=`uname -m`".

FUJITSU PCI GigabitEthernet 3.0 Update1 or newer

${MNT}/fjgi/install -R /a -d ${MNT}/fjgi -p "$PF_ARCH"

FUJITSU PCI GigabitEthernet 4.0 or newer

${MNT}/fjgi/install_v4 -R /a -d ${MNT}/fjgi -p "$PF_ARCH"

FUJITSU ULTRA LVD SCSI Host Bus Adapter Driver

${MNT}/fjulsa/install -R /a -d ${MNT}/fjulsa -p "$PF_ARCH"

Finish script sample setting

#!/bin/sh

### Edit here ###

JUMPSTART_HOST=

JUMPSTART_DIR=/jumpstart

### End of edit ###

### MAIN ###

MNT=/a/mnt

mount -F nfs ${JUMPSTART_HOST}:${JUMPSTART_DIR} ${MNT}

PF_ARCH=`uname -m`

${MNT}/fjgi/install -R /a -d ${MNT}/fjgi -p "$PF_ARCH"

41

${MNT}/fjulsa/install -R /a -d ${MNT}/fjulsa -p "$PF_ARCH"

${MNT}/FJPFCA/bin/pfcapkgadd.sh -R /a -p "$PF_ARCH"

# Copy fjpfca.conf

if [ -f /kernel/drv/fjpfca.conf ]

then

echo "copying fjpfca.conf "

cp /kernel/drv/fjpfca.conf /a/kernel/drv/fjpfca.conf

COPY_STATUS="$?"

if [ "$?" != "0" ]

then

echo "ERROR: fjpfca.conf copy failed."

fi

else

echo "NOTICE: /kernel/drv/fjpfca.conf does not exists."

fi

## Copy sd.conf

if [ -f /kernel/drv/sd.conf ]

then

echo "copying sd.conf "

cp /kernel/drv/sd.conf /a/kernel/drv/sd.conf

COPY_STATUS="$?"

if [ "$?" != "0" ]

then

echo "ERROR: sd.conf copy failed."

fi

else

echo "NOTICE: /kernel/drv/sd.conf does not exists."

fi

umount ${MNT}

7. Edit the rules file

Edit the /jumpstart/rules file with a text editor. Specify a profile and finish script used for eachhost in the rules file.

42

The rules file comes with a number of sample settings by default. Comment them out, becausethey are not required.

Append the following to the rules file:

hostname <install machine hostname> - profile finish

8. Check and enable the rules file

Execute the check command to enable the rules file.


(INSTALL SERVER) # cd /jumpstart <RETURN>

(INSTALL SERVER) # /jumpstart/check -p /export/install/Solaris10_hostname -r rules<RETURN>



(INSTALL SERVER) # /jumpstart/check -p /tmp/media -r rules <RETURN>

If the check command is executed and the following error messages are displayed, thecheck command execute again after the following procedures.

Error message:

ERROR: /tmp/media is not a valid Solaris 2.x CD image

(INSTALL SERVER) # cd /tmp/media/Solaris_10/Tools/Boot <RETURN>

(INSTALL SERVER) # bzcat lu.cpio.bz2 | cpio -idum <RETURN>

(INSTALL SERVER) # ls usr/sbin/install.d/chkprobe <RETURN>

usr/sbin/install.d/chkprobe


(INSTALL SERVER) # /jumpstart/check -p /tmp/media -r rules <RETURN>

9. Enter network boot settings.

The setting varies with each installation target machine model.


(INSTALL SERVER) # /export/install/Solaris10_hostname/Solaris_10/Tools/add_install_client -i install machine IP address -e install machine mac address -s installserver host name:/export/install/Solaris10_hostname -c install server host name:/jumpstartinstall machine host name sun4v <RETURN>


43

(INSTALL SERVER) # /export/install/Solaris10_hostname/Solaris_10/Tools/add_install_client -i install machine IP address -e install machine mac address -s installserver host name:/export/install/Solaris10_hostname -c install server host name:/jumpstartinstall machine host name sun4u <RETURN>

4.1.1.5 Configuring the Fibre Channel boot code

Configure the Fibre Channel boot code required to boot the OS.

Perform this procedure on the install machine.

1. If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute thefollowing command:

ok setenv auto-boot? false <RETURN>

ok reset-all <RETURN>

If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switchto service mode and execute the following command:


2. Make sure that Fibre Channel card is identified on the OBP. Check the physical path name of the slotin which the Fibre Channel card is mounted.

Example of having a single-channel 4Gbps Fibre Channel card (SE0X7F11x) and a dual-channel4Gbps Fibre Channel card (SE0X7F12x) mounted on a server

ok show-devs <RETURN>

/pci@1,700000

/pci@2,600000

**

/openprom

/chosen

/packages

/pci@1,700000/fibre-channel@0 *physical path name of single-channel 4Gbps FibreChannel card

/pci@2,600000/fibre-channel@0 *physical path name of dual-channel 4Gbps FibreChannel card port0

/pci@2,600000/fibre-channel@0,1 *physical path name of dual-channel 4Gbps FibreChannel card port1

/mc@0,0/bank@0,c0000000

/mc@0,0/bank@0,80000000

3. With the boot code enabled on the Fibre Channel card used for booting the OS, restart the server.Move to the Fibre Channel card physical path (/pci@1,700000/fibre-channel@0) confirmed in Step 2

44

before executing a setting command. There is no need to enable the boot code on those Fibre Channelcards that do not use the boot feature.

ok cd /pci@1,700000/fibre-channel@0 <RETURN> Move to the Fibre Channel cardphysical path

ok ENABLE fjpfca-set-bootfunction <RETURN> Enable the boot feature.

ok reset-all <RETURN> Initialize the server.

Perform this step on all cards used for booting the OS.

4. After restarting the server, view information about the disk array devices connected to it.

Example: fabric connection

ok cd /pci@1,700000/fibre-channel@0 <RETURN>

ok PROBE fjpfca-info <RETURN>

Target - DID 10500 210000e00040101d9 FUJITSU-E4000-0000

Target - DID 10600 210000e00040101da FUJITSU-E4000-0000

Targets residing on a fabric connection may appear as "-," but this is of no concern.

5. Configure the disk array device to make it identifiable with the Fibre Channel boot code.

There are two ways to make a disk array device identifiable on a fabric connection (using the FCSwitch): automatic setting and manual setting. Automatic setting for making a disk array deviceidentifiable, when selected, offers the following benefits:

(1) If multiple hosts use a disk array device as a boot device each, they still can share and use theSolaris OS install image on the install server.(2) The Fibre Channel driver is made easier to configure.

These two methods of identifying a disk array device are described below.

a. [Automatic setting]Remove the disk array device setting in the Fibre Channel boot code. Execute thefollowing command:


ok ENABLE fjpfca-all-target-cancel <RETURN>

fjpfca-all-target-cancel: Delete bind target parameter ...

If a disk array device is identified through automatic setting, set zoning with the FCSwitch to disable a Fibre Channel card from connecting to multiple disk array devices. Inan environment in which a Fibre Channel card is allowed to connect to multiple disk arraydevices, allow disk array devices to be identified through manual setting.

b. [Manual setting]In the Fibre Channel switch (fabric connection) environment, execute the fjpfca-bind-

target command to define a disk array device that is identifiable to the Fibre Channel bootcode.

45

The WWPN displayed in Step 4 or the DID value is required at this time. There is no needto execute the fjpfca-bind-target command in the FC-AL environment, because the diskarray device is configured automatically.

Move to the Fibre Channel card physical path (/pci@1,700000/fibre-channel@0) confirmed inStep 2 before carrying out a configuration command.

In case the target ID 16 is to be specified in fjpfca.conf, specify "10" which is 16 in hexadecimalnotation.

o Definition by WWPN


ok 10 target-wwpn 210000e0004101d9 fjpfca-bind-target <RETURN>

fjpfca-bind-target: Change bind target parameter

o Definition by DID


ok 10 target-did 10500 fjpfca-bind-target <RETURN>

fjpfca-bind-target: Change bind target parameter .

* The setting is not required in the FC-AL (Private loop) environment.

6. Other settings

The connection topology, link speed, and server and disk array device power supply interlock waittime can be modified as required. The connection topology and link speed are set to "auto"(automatic setting) by default.

For more details, see Appendix A, "Boot Device Setup Commands."

For example, if automatic setting is not used on a connection with a Fibre Channel switch readyfor 2Gbps, enter the following:

ok 2g fjpfca-set-linkspeed <RETURN>

ok nport fjpfca-set-topology <RETURN>

Confirm the settings.

ok fjpfca-output-prop <RETURN>

boot function: ENABLE

topology : N_Port

link-speed : 2G

boot wait time: DISABLE (interval time: DISABLE/ boot wait msg: DISABLE)

bind-target: Target_ID=16,WWN=0x210000c0004101d9

46

For example, if a power supply interlocking boot wait time of 1200 seconds (20 minutes) is set ona direct connection with a disk array device ready for 2Gbps, enter the following:


ok al fjpfca-set-topology <RETURN>

ok d# 1200 fjpfca-set-boot-wait-time <RETURN>

Confirm the settings.


boot function: ENABLE

topology : AL

link-speed : 2G

boot wait time: 1200 sec (interval time: DISABLE/ boot wait msg: DISABLE


7. Execute the following command to reset:


4.1.1.6 Executing network installation

Execute the following command on the OBP of the installation target host:

ok boot net - install <RETURN>

Subsequently, proceed with the install process as directed by onscreen guidance.

After OS installation is complete and the OBP <ok> prompt is displayed, perform the proceduresdescribed in section "4.2 Making the Path to a Boot Disk Redundant".

If Solaris 10 is installed on a server that supports a graphics card to use a bitmapped display, right-clickanywhere on the screen to display a menu, from which open a terminal to migrate the system to the OBPenvironment and proceed to the Section 4.2, "Making the Path to a Boot Disk Redundant" procedure.The message "Click <Reboot> to continue" is displayed, but ignore it for now.

47

4.1.2 Creating a boot disk by copying an existing bootdisk residing on an internal disk

This section explains how to copy a boot disk that has already been created on an internal disk orelsewhere and use it to create a boot disk on a disk array device. Before getting started, make sure thatthe server has been started from the OS stored on the internal disk and a connection with the disk arraydevice has been configured via a Fibre Channel card.

A single-path connection with the disk array device will do for now. Convert the connection to amultipath implementation according to Section 4.2, "Making the Path to a Boot Disk Redundant."

Perform this procedure entirely from the install machine console.

This procedure does not allow a boot disk to be created on a target device that has been connected by theauto-target bind feature of the Fibre Channel driver. Be sure to configure a target device with fcp-bind-target to create a boot disk on it.

For more information about configuring fcp-bind-target, refer to "FUJITSU PCI Fibre Channel Guide."

48

4.1.2.1 Getting ready to copy the boot disk to a disk arraydevice

1. Execute the format command or the like to confirm the disk array device on which to create a bootdisk.

# format <RETURN>

Searching for disks...done

AVAILABLE DISK SELECTIONS

0. c7t16d0 <FUJITSU-ETERNUS-4000 cyl 1038 alt 2 hd 64 sec 256>

/pci@1,700000/fibre-channel@0/sd@10,0

Check for the availability of sufficient disk space at the boot disk creation destination andsufficient associated partition space when compared with the source to copy from. If a slice hasnot been created in the LUN at the boot disk creation destination, execute the format commandto create one.

4.1.2.2 Creating a boot diskThe boot disk is created referring to the following procedures.

Procedure for copying internal disk (UFS file system) to disk array device (UFS file system).

1. Migrate the system to the OBP environment.


Launch the system in single-user mode.

ok boot -s <RETURN>

2. Write a boot block.

Write a boot block to the system block (using a path that has been verified by carrying out theformat command or the like).

# installboot /usr/platform/`uname -m`/lib/fs/ufs/bootblk /dev/rdsk/c7t16d0s0 <RETURN>

3. Create a file system.

# newfs -v /dev/rdsk/c7t16d0s0 <RETURN>

4. Execute the mount command (mount the LUN for the boot disk on the identified disk array device).

# mount -F ufs /dev/dsk/c7t16d0s0 /mnt <RETURN>

49

5. Copy the boot disk to the disk array device.

# ufsdump 0f - /dev/rdsk/c0t0d0s0 | ( cd /mnt; ufsrestore rf -) <RETURN>

The command above, when carried out, copies data located in directories other than /mnt to RAID.

6. Copy other partitions.

If /var and /opt have been defined as separate partitions, repeat Steps 3 to 5 above for eachpartition.

4.1.2.3 Editing mount table information

Enter the access path to the disk array device on which a boot disk has been created in /mnt/etc/vfstab.

All the settings of the access paths to LUN that makes the boot disk are added, comment out access pathsto LUN other than the internal disk and the boot disk.

#device device mount FS fsck mount mount

#to mount to fsck point type pass at boot options

#

#/dev/dsk/c0t0d0s3 - - swap - no -

#/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no-

#/dev/dsk/c0t0d0s1 /dev/rdsk/c0t0d0s1 /var ufs 1 no-

fd - /dev/fd fd - no -

/proc - /proc proc - no -

/dev/dsk/c7t16d0s3 - - swap - no -

/dev/dsk/c7t16d0s0 /dev/rdsk/c7t16d0s0 / ufs 1 no-

/dev/dsk/c7t16d0s1 /dev/rdsk/c7t16d0s1 /var ufs 1 no-

..

If a disk array device has a LUN other than its boot disk defined in sd.conf, remove the definition from/mnt/kernel/drv/sd.conf.

4.1.2.4 Configuring the Fibre Channel boot code

Configure the Fibre Channel boot code to allow the OS to be booted from a disk array device.

Example: Set on fjpfca0.

50

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -b ENABLE <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -c /kernel/drv/fjpfca.conf <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -v <RETURN>

boot_function : ENABLE

topology : N_Port

link-speed : 4G

boot wait time : DISABLE ( interval time : DISABLE , boot wait msg : DISABLE )


4.1.2.5 Resetting the server

If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute the followingcommands depending on the system status.

The OS is booting

execute the following command:

# /usr/sbin/eeprom auto-boot?=false <RETURN>


After migrating to the OBP environment, execute the following command:


In the OBP environment

Execute the following command:



The server is turned off

Turn on the server and execute “The OS is booting” or “In the OBP environment” depending onthe system status.

If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000 set the server mode switch toservice mode and execute the following commands depending on the system status.

The OS is booting

Set the server mode switch to service mode and execute the following command:

51


After migrating to the OBP environment, execute the following command:


In the OBP environment

Set the server mode switch to service mode and then execute the following command:


The server is turned off

Set the server mode switch to service mode and then turn on the server.

4.1.2.6 Booting from a disk array device

Specify a disk array device from which boot the OS.

ok boot /pci@1,700000/fibre-channel@0/disk@10,0 <RETURN> (*1)

Boot device: /pci@1,700000/fibre-channel@0/disk@10,0 File and args: kernel/sparcv9/unix

(*1) The value (in the example above, "10,0") that follows "disk" specified by boot denotes thetarget_id/LUN. It must be the same as the value of target_id/LUN of the disk array device that isidentified by the Fibre Channel driver after the OS boots. In the FC-AL environment, specify thevalue of target_id that is displayed when fjpfca-info is executed.

The values of target_id and LUN need be specified in hexadecimal at boot time.

52

4.2 Making the Path to a Boot Disk RedundantThis section explains how to make the path to a boot device based on the ETERNUS Multipath Driverredundant.

4.2.1 Applying the FUJITSU Recommended & SecurityPatches Cluster and PTF and installing theEnhanced Support Facility

Apply FUJITSU Recommended & Security Patches Cluster and PTF, if it is yet to be applied. InstallEnhanced Support Facility, if it is yet to be installed. If it is already applied and installed, proceed to thenext procedure.

1. Specify the disk array device and boot the OS in single-user mode.

ok boot /pci@1,700000/fibre-channel@0/disk@10,0 -s <RETURN>


2. Install FUJITSU Recommended & Security Patches Cluster as instructed in "CLUSTER_README",install Enhanced Support Facility as instructed in "Enhanced Support Facility Install Guide" and installPTF as instructed in "PTF Guide."

3. Restart the host in the following way:


4. After migrating to the OBP environment, execute the following command:


5. Specify the disk array device and boot the OS.

ok boot /pci@1,700000/fibre-channel@0/disk@10,0 <RETURN>


53

4.2.2 Configuring the ETERNUS Multipath Driver

This section explains how to configure the ETERNUS Multipath Driver to make the path to a boot deviceredundant.

About target ID of the equipment used as a boot device, it recommends setting it as the same value withtwo Fibre Channel cards which constitute a multipath.

4.2.2.1 Single system (non-cluster system)1. Specify a disk array device and boot the OS from it to install the ETERNUS Multipath Driver.

Launch the host from the boot disk on the disk array device. Then, install ETERNUS MultipathDriver as instructed in the "ETERNUS Multipath Diver Install Guide." When the install completes,respond with "y" at the following prompt to let the grmpdautoconf command execute automatically toproceed to the multipath building process in Step2.

Do you want to make a multipath configuration now?

If the ETERNUS Multipath Driver package has already been installed, execute grmpdautoconfto proceed to the multipath building process in Step 2.

# /usr/sbin/grmpdautoconf <RETURN>

2. Execute grmpdautoconf to build a multipath.Work with grmpdautoconf interactively. For more details, refer to "ETERNUS Multipath DriverUser's Guide." During this interactive session, make the following choices:

Select "Manual selection" m in response to the automatic/manual path selection prompt.

Select an access path automatically or manually?

** If automatic selection is selected, all access paths marked "New" are registered with thesystem.

** Select automatic selection if an access path has been properly selected at ETERNUS andswitch setup.

** All paths marked "Exist" are eligible for LUN installation.

** All devices residing on an AL connection are eligible for LUN and device installation.

Manual selection ---> 'm'

Automatic selection ---> 'a'

Quit ---> 'q'

Enter a key. [m,a,q] m <RETURN>

54

Different manual path selection screens are displayed depending on which method of diskarray device identification by the Fibre Channel driver has been selected, automatic settingor manual setting.

a. [Automatic setting]

Select a start-up path on the manual path selection screen.

Adapter Switch ETERNUSStatus

instance WWN WWN product

-----+-------------------------------------+-----+------------------------------------------------------+-----

[ ] 1 fjpfca0 100000000e24ac06 1 210000e0004101d9 E4000 CM1CA0P0New

[ ] 2 fjpfca1 100000000e244737 3 210000e0004101da E4000 CM0CA0P0New

Among the paths marked "New," enter the number of the access path you wantregistered with the system.

** The access path list is redisplayed after the entry.

** Selected paths are marked with "*."

** If an invalid number is selected, enter that number again to deselect it.


** All devices residing on an AL connection are eligible for LUN and deviceinstallation.

Enter path numbers ---> Enter numerals (multiple path numbers can be enteredseparated from each

other with a comma)

Conclude entry ---> 'x'

Quit ---> 'q'

Enter a key. [Path number ,x,q] 1 2 <RETURN>



-----+-------------------------------------+-----+------------------------------------------------------+-----

[*] 1 fjpfca0 100000000e24ac06 1 210000e0004101d9 E4000 CM1CA0P0New

[*] 2 fjpfca1 100000000e244737 3 210000e0004101da E4000 CM0CA0P0New

55


** The access path list is redisplayed after the entry.** Selected paths are marked with "*."





other with a comma)


Quit ---> 'q'

Enter a key. [Path number ,x,q] x <RETURN>

The setting of the path (disk array device) selected above is reflected in the Fibre Channel driversetting file (/kernel/drv/fjpfca.conf). Once this step completes, the setting of the disk array devicethat is identified by the Fibre Channel driver is fixed, so the following setting in the Fibre Channeldriver setting file can be removed:


b. [Manual setting]

The wwn entered in fjpfca.conf appears as "Exist" or "AL." Leave all other paths unselected,selecting "Confirmed (x)" for them.



-----+----------------------------------+-----+----------------------------------------------------+-----

fjpfca0 100000000e24ac06 1 210000e0004101d9 E4000 CM1CA0P0Exist





56





other with a comma)


Quit ---> 'q'

Enter a key. [Path number ,x,q] 1 <RETURN>



-----+----------------------------------+-----+------------------------------------------------------+------










other with a comma)


Quit ---> 'q'


When SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440 is being used, itis necessary to choose the mode of an access path. Select mplb mode at the prompt for access

57

path mode selection. The ETERNUS Multipath Driver supports two different multipathaccess modes: Solaris standard mode, in which the multipath is accessed from a Solarisstandard special file, and mplb mode, in which the multipath is accessed from a mplb specialfile as in the past.

In the Solaris 10 environment, select mplb mode for the access path mode. Solaris standard modedoes not work in the SAN Boot environment.

When SPARC Enterprise M3000/M4000/M5000/M8000/M9000 is being used, it is not necessaryto choose access path mode.

Choose an access special file between the following:

Solaris standard special file (/dev/[r]dsk/c*t*d*s*)

mplb special file (/dev/FJSVmplb/[r]dsk/mplb*s*)

/dev/[r]dsk/c*t*d*s* ---> 's'

/dev/FJSVmplb/[r]dsk/mplb*s* ---> 'm'

Enter a key. [s, m] m <RETURN>

3. Check the device path name of the boot device. The grmpdautoconf command carried out in Step 2displays a combination of a multipath management special file and a selected access special file. Usethe output of the ls command to identify the boot disk and the physical device path name of eachconfiguration path. Physical device path names thus found are used in Steps 6 and 9.

# ls -l <Boot disk slice 0 > <RETURN>

# ls -l < Configuration path slice 2 > <RETURN>

Assume that grmpdautoconf has delivered the following output listing:

*** Phase 1: read mplb.conf ***

*** Phase 2: read /dev ***

*** Phase 3: read /devices ***

*** Phase 4: compare mplb.conf and /devices ***

Path : Action : Element path : LUN : Storage

mplb0 : new : c2t16d0s2 c13t16d0s2 : 0 : E40004641- 130011 :



The boot disk and the paths that configure it up are as follows:

Boot disk /dev/FJSVmplb/rdsk/mplb0s0Configuration path /dev/rdsk/c2t16d0s2

58

/dev/rdsk/c13t16d0s2

Execute the ls command to check the device path names.

# ls -l /dev/FJSVmplb/rdsk/mplb0s0 <RETURN>

lrwxrwxrwx 1 root root 36 Aug 29 12:05 /dev/FJSVmplb/rdsk/mplb0s0 -> (line wrapping)

../../../devices/pseudo/mplb@0:a,raw <RETURN>

^^^^^^^^^^^^^^^

# ls -l /dev/rdsk/c2t16d0s2 <RETURN>

lrwxrwxrwx 1 root root 58 Aug 29 17:13 /dev/rdsk/c2t16d0s2 -> (line wrapping)

../../devices/pci@1,700000/fibre-channel@0/mplbt@10,0:c,raw <RETURN>

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^




^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

4. Set a Fibre Channel driver link speed.

Edit the Fibre Channel driver setting file (/kernel/drv/fjpfca.conf) to set a Fibre Channel linkspeed.

In configuring the Fibre Channel driver, the link speed setting can be set to automatic selection forthe sake of easier connectivity. The expected link speed may not be attained depending on theconnection status. Therefore, set the highest transmission rate available in the environment.


port=


For more information about configuring fjpfca.conf, refer to "FUJITSU PCI Fibre ChannelGuide."

5. Load the Fibre Channel boot codes that are used to access the boot disk with a disk array device bootsetting.NoteThe setting to the Fibre Channel boot code used by boot is unnecessary now.






59

topology : N_Port

link-speed : 4G


bind-target: Target_ID=16,WWN=0x210000e0004101d9

6. Correct the system settings to meet the multipath implementation.a. Root device setting (/etc/system)

Edit /etc/system file to set rootdev and forceload. For the rootdev setting, set the boot diskphysical device name as found in Step 3 above, excluding "../../devices" at the beginning and",raw" at the end.

When the setting concerning forceload to each driver exists in /etc/system file, an additionalsetting need not be done.

rootdev: /pseudo/mplb@0:a

forceload: drv/mplbt

forceload: drv/mplb

forceload: drv/sd

b. Mount information setting (/etc/vfstab)

Edit /etc/vfstab file to rewrite each entry to a path name after the multipath implementation.

/dev/FJSVmplb/dsk/mplb0s0 /dev/FJSVmplb/rdsk/mplb0s0 / ufs 1 no -

/dev/FJSVmplb/dsk/mplb0s3 - - swap - no -

c. An access setup of a boot disk (/kernel/drv/sd.conf).

When target ID of the equipment used as a boot device has differed with two Fibre Channel cardswhich constitute a multipath, target ID about all the paths of a boot disk may not be written tosd.conf. In that case, the following setup is needed.

○ Edit /kernel/drv/sd.conf

The definition of target ID of the path used as a boot disk is added.

Example: The definition of "Target ID = 18" is added.

name=”sd” class=”scsi” target=18 lun=0;

○ Reconstruction of sd driver

# touch /reconfigure <RETURN>

or

# update_drv -f sd <RETURN>

60

7. Configure a dump device as required.

# dumpadm -d /dev/FJSVmplb/dsk/mplb0s3 <RETURN>

8. Stop the unit and reset the OBP environment.



9. Configure a boot device.

Configure a boot device on all redundant paths to the boot disk on the OBP. Take the physicaldevice name of each configuration path found in Step 3, excluding "../../devices" at the beginningand ":*,raw" at the end, and set it with "mplbt" being replaced with "disk."

ok nvalias raid1 /pci@1,700000/fibre-channel@0/disk@10,0 <RETURN>


ok setenv boot-device raid1 raid2 <RETURN>

10. If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute thefollowing command;

ok setenv auto-boot? true <RETURN>

If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switch backto AUTO.

11. Start the host.

ok boot <RETURN>

4.2.2.2 Cluster system

Perform this procedure to build a cluster system using PRIMECLUSTER.

1. Specify a disk array device and boot the OS from it to install the ETERNUS Multipath Driver.

Launch the host from the boot disk on the disk array device. Then, install ETERNUS MultipathDriver as instructed in "ETERNUS Multipath Driver Install Guide." When the install completes,respond with "y" at the following prompt to let the grmpdautoconf command executeautomatically to proceed to the multipath building process in Step 2.

Do you want to make a multipath configuration now ?

If the ETERNUS Multipath Driver package has already been installed, execute grmpdautoconf toproceed to the multipath building process in Step 2.

61

# /usr/sbin/grmpdautoconf <RETURN>

2. Execute grmpdautoconf to build a multipath.

Work with grmpdautoconf interactively. For more details, refer to "ETERNUS Multipath DriverUser's Guide." During this interactive session, make the following choices:

Select "Manual selection" m in response to the automatic/manual path selection prompt.

Select an access path automatically or manually?

** If automatic selection is selected, all access paths marked "New" are registered with thesystem.

** Select automatic selection if an access path has been properly selected at ETERNUS andswitch setup.


** All devices residing on an AL connection are eligible for LUN and device installation.

Manual selection ---> 'm'

Automatic selection ---> 'a'

Quit ---> 'q'

Enter a key. [m,a,q] m <RETURN>

Different manual path selection screens are displayed depending on which method of diskarray device identification has been selected in the Fibre Channel driver, automatic settingor manual setting.

a. [Automatic setting]

Select a startup path on the manual path selection screen.



-----+----------------------------------+-----+--------------------------------------------------+-----

[ ] 1 fjpfca0 100000000e24ac06 1 210000e0004101d9 E4000 CM1CA0P0New






62




other with a comma)


Quit ---> 'q'

Enter a key. [Path number ,x,q] 1 2 <RETURN>

Adapter Switch ETERNUS Status


-----+----------------------------------+-----+-------------------------------------------------+-----

[*] 1 fjpfca0 100000000e24ac06 1 210000e0004101d9 E4000 CM1CA0P0 New









other with a comma)


Quit ---> 'q'


The setting of the path (disk array device) selected above is reflected in the Fibre Channel driversetting file (/kernel/drv/fjpfca.conf). Once this step completes, the setting of the disk array devicethat is identified by the Fibre Channel driver is fixed, so the following setting in the Fibre Channeldriver setting file can be removed:


63

b. [Manual setting]

The wwn entered in fjpfca.conf appears as "Exist" or "AL." Leave all other paths unselected,selecting "Confirmed (x)" for them.



---+-----------------------------------+-----+------------------------------------------------------+-----










other with a comma)


Quit ---> 'q'

Enter a key. [Path number ,x,q] 1 <RETURN>



-----+----------------------------------+-----+------------------------------------------------------+-----




64







other with a comma)


Quit ---> 'q'


Select y in response to the prompt asking whether to use PRIMECLUSTER.

Is the disk array device used as a PRIMECLUSTER/SafeCLUSTER shared disk?

**If PRIMECLUSTER/SafeCLUSTER is used, use the multipath setup feature provided bythe relevant product.

** The value of maxthrottle also needs to be reviewed.

Yes ---> 'y' (Processing ends)

No ---> 'n'

Enter a key. [y,n] y <RETURN>

Processing up to sd has ended successfully.

*** IMPORTANT NOTICE ***

Installation of ETERNUS Multipart Driver Package was successful.

3. Next, execute the mplbconfig command.

# /usr/sbin/mplbconfig -o /tmp/mplb-file1 <RETURN>

*** Phase 1: Loading mplb.conf ***

*** Phase 2: Checking the device file in /dev ***

*** Phase 3: Checking the device fie in /devices ***

*** Phase 4: Checking mplb.conf against the configuration in /devices ***

65

=== Multipath configuration plan ===

Existing instance : 0

New instance : 2

Add a path : 0 (instance)

Delete a path : 0 (instance)

4. Delete all lines except for the system volume. Edit /tmp/mplb-file1 using a vi editor or the like todelete all except for the path to a system disk.

With a cluster system, the instance number of each local multipath disk, such as a boot disk, mustnot be defined in duplicate among the nodes that configure the cluster. Change instance numbers(designated by X in mplbX) between 0 and 2047 to avoid duplication with other nodes.

*** mplb config file ***

Path : Process : Configuration path : LUN : Device information

mplb0 : new : c2t16d0s2 c13t16d0s2 : 0 : E40004641- 130011

5. Apply the edited file, converting the system volume into a multipath implementation.

# /usr/sbin/mplbconfig -f /tmp/mplb-file1 <RETURN>

*** Phase 1: Loading mplb.conf ***

*** Phase 2: Checking the device file in /dev ***

*** Phase 3: Checking the device file in /devices ***

*** Phase 4: Checking mplb.conf against the configuration in /devices ***

*** Phase 5: Updating mplb.conf ***

=== Multipath configuration plan ===

Existing instance: 0

New instance: 1

Add a path : 0 (instance)

Delete a path : 0 (instance)

6. Check the device path name of the boot device. The grmpdautoconf command carried out in Step 4displays a combination of a multipath management special file and a selected access special file. Usethe output of the ls command to identify the boot disk and the physical device path name of eachconfiguration path. Physical device path names thus found are used in Steps 9 and 12.

# ls -l <Boot disk slice 0 > <RETURN>

# ls -l <Configuration path slice 2 > <RETURN>

66

Assume that grmpdautoconf has delivered the following output listing:

*** Phase 1: read mplb.conf ***

*** Phase 2: read /dev ***

*** Phase 3: read /devices ***

*** Phase 4: compare mplb.conf and /devices ***

Path : Action : Element path : LUN : Storage




The booth disk and the paths that configure it up are as follows:

Boot disk /dev/FJSVmplb/rdsk/mplb0s0

Configuration path /dev/rdsk/c2t16d0s2

/dev/rdsk/c13t16d0s2

Execute the ls command to check the device path names.

# ls -l /dev/FJSVmplb/rdsk/mplb0s0 <RETURN>

lrwxrwxrwx 1 root root 36 Aug 29 12:05 /dev/FJSVmplb/rdsk/mplb0s0 -> (line wrapping)

../../../devices/pseudo/mplb@0:a,raw <RETURN>

^^^^^^^^^^^^^^^




^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^




^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

7. Set a Fibre Channel driver link speed.

Edit the Fibre Channel driver setting file (/kernel/drv/fjpfca.conf) to set a Fibre Channel driverlink speed.

67

In configuring the Fibre Channel driver, the link speed setting can be set to automatic selection forthe sake of easier connectivity. The expected link speed may not be attained depending on theconnection status. Therefore, set the highest transmission rate available in the environment.


port=


For more information about configuring fjpfca.conf, refer to the "FUJITSU PCI Fibre ChannelGuide."

8. Load the Fibre Channel cards that are used to access the boot disk with a disk array device boot setting.NoteThe setting to the Fibre Channel boot code used by boot is unnecessary now.






topology : N_Port

link-speed : 4G


bind-target: Target_ID=16,WWN=0x210000e0004101d9

9. Correct the system settings to meet the multipath implementation.a. Root device setting (/etc/system)

Edit /etc/system file to set rootdev and forceload. For the rootdev setting, set the boot diskphysical device name as found in Step 6 above, excluding "../../devices" at the beginning and",raw" at the end.

When the setting concerning forceload to each driver exists in /etc/system file, an additionalsetting need not be done.



forceload: drv/mplb

forceload: drv/sd

b. Mount information setting (/etc/vfstab)

Edit /etc/vfstab file to rewrite each entry to a path name after the multipath implementation.

68

/dev/FJSVmplb/dsk/mplb0s0 /dev/FJSVmplb/rdsk/mplb0s0 / ufs 1 no -

/dev/FJSVmplb/dsk/mplb0s3 - - swap - no -

c. An access setup of a boot disk (/kernel/drv/sd.conf).

When target ID of the equipment used as a boot device has differed with two Fibre Channel cardswhich constitute a multipath, target ID about all the paths of a boot disk may not be written tosd.conf. In that case, the following setup is needed.

○ Edit /kernel/drv/sd.conf

The definition of target ID of the path used as a boot disk is added.

Example: The definition of "Target ID = 18" is added.


○ Reconstruction of sd driver


or


10. Configure a dump device as required.

# dumpadm -d /dev/FJSVmplb/dsk/mplb0s3 <RETURN>

11. Stop the equipment and reset the OBP environment.



12. Configure a boot device.

Configure a boot device on all redundant paths to the boot device on the OBP. Take the physicaldevice name of each configuration path found in Step 6, excluding "../../devices" at the beginningand ":*,raw" at the end, and set it with "mplbt" being replaced with "disk."



ok setenv boot-device raid1 raid2 <RETURN>

13. If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute thefollowing command;


If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switchback to AUTO.

69

14. Start the host.

ok boot <RETURN>

70

4.3 Boot Disk MirroringThis section explains how to mirror between multipath implementations of two boot disks while the OS isbooted from one of them.

4.3.1 Mirroring by PRIMECLUSTER GDS

1. Verify that zoning has been implemented by the FC Switch as shown above.2. Verify that the connection of the Fibre Channel card to the disk array device at the mirroring

destination has been configured properly in the OBP environment.



Target -- DID 10500 210000e00040101d9 FUJITSU-E4000-0000

Target -- DID 10600 210000e00040101da FUJITSU-E4000-0000

3. Boot the OS and verify that the connection of the Fibre Channel driver to the disk array device at themirroring destination has been configured properly.

# /usr/sbin/FJSVpfca/fc_info -p <RETURN>

adapter=fjpfca#0 :

port_id=0x010500 tid=0 wwn=210000e00040101d9 adapter=fjpfca#1 connected

class=class3

port_id=0x01060 tid=0 wwn=210000e00040101da adapter=fjpfca#1 connected

class=class3

71

4. If a Multipath Driver is yet to be set on the disk array device at the mirroring destination, set it.Referring to "ETERNUS Multipath Driver User's Guide," set a Multipath Driver and create amultipath disk on the disk array device (ETERNUS 2) at the mirroring destination.

5. If using the ETERNUS Multipath Driver, add a definition of the mirroring destination disk to targetdriver setting file /kernel/drv/sd.conf.

Example: Identify target ID 16, logical unit 1.


6. Execute the install with reference to the PRIMECLUSTER GDS manual.

7. Refer to “PRIMECLUSTER Global Disk Services Guide” and mirror the disk at the mirroring sourceand destination with each other.

If a system disk is mirrored using PRIMECLUSTER GDS, the message below may bedisplayed at boot time. This message may be ignored.

NOTICE: “forceload: drv/<driver name> appears more than once in /etc/system.

* One of mplb, mplbt and sd appears in place of <driver name>.

This message is displayed if the setting forceload is defined in duplicate in /etc/system file. Toleave this message hidden, delete later occurrences of the setting of forceload.

forceload: drv/mplb

~

forceload: drv/mplb Delete this line.

8. If a snapshot of a system volume is created using PRIMECLUSTER GDS Snapshot, perform Step 1through 5 above for the snapshot disk the same way as for the disk at the mirroring destination. Then,configure the snapshot as instructed in “PRIMECLUSTER Global Disk Service Guide.”

72

4.3.2 Notes on using PRIMECLUSTER

4.3.2.1 Cluster system building procedure

Build a cluster system in the procedural steps described below.

4.3.2.2 Switching cluster system at the time of boot diskfailures

When the boot disk cease to be accessed on a disk array device, the abnormality detection feature ofPRIMECLUSTER may not work, depending on the state of the system.

As the operation of a node that cannot access the boot disk becomes irregular, use the followingprocedure to cause panic on the node.

[In the case where login to a node other than the corresponding node is possible]

Stop the corresponding node with the "sdtool" command.

# sdtool -k <corresponding node> <RETURN>

< Reference >

For details on the sdtool(1M) command, refer to the manual page of sdtool(1M) command.

[In the case when login to any node is impossible]

Cause panic on the corresponding node. For details on the procedure, refer to the server's operationmanual.

73

Appendix A Boot Device Setup Commands

This appendix focuses on the commands used to configure the boot code on the Fibre Channel card.These commands are executable on either the OS or the OBP.

The commands introduced here work only on the single-channel 4Gbps Fibre Channel card (SE0X7F11x)and dual-channel 4Gbps Fibre Channel card (SE0X7F12x).

A.1 Command Executable on the OSPerform this procedure with the OS being booted and with FUJITSU PCI Fibre Channel 4.0 or any newerpackage installed.

1 fc_hbaprp

Name

fc_hbaprp

Format

/usr/sbin/FJSVpfca/fc_hbaprp -i adpname -f tgt_id -P WWN

-f tgt_id -I PORT_ID

-d tgt_id

-D [-y]

-w boot-wait-time

-l linkspeed

-t topology

-v

-s savefile

-r|-R filename

-c conffile

-C [-y]

-b ENABLE|DISABLE

74

Function

Configures the boot code on the Fibre Channel card.

Operands

The settings that work on the boot code on the Fibre Channel card are listed below.

All these settings need to be accompanied by the specification of -i adpname. Specify the instancename of the Fibre Channel driver as adpname.

-i adpname -f tgt_id -P WWN

-i adpname -f tgt_id -I PORT_ID

Configure a target device. Up to 10 entries can be registered.

The following values can be set:

tgt_id Specifies Target_ID of the target device with a decimal number.

WWN Specifies WWPN of the target device with a hexadecimal number (boot devicespecification by WWPN).

PORT_ID Specifies Port_ID(DID) of the target device with a hexadecimal number (boot devicespecification by Port_ID).

-i adpname -d tgt_id

Erases the setting of the target device. The following value can be set:

tgt_id Specifies Target_ID of the target device with a decimal number.

-i adpname -D [-y]

Erases all settings of the target device.

If -y is not attached, a message asking if you are sure you want to erase the settings is displayed.

If -y is attached, the settings are erased unconditionally.

-i adpname -w boot-wait-time

Sets a boot wait time in seconds. The following value can be set:

boot-wait-time Specifies a boot wait time with a decimal number. Either 0 second (no boot-wait-time) or a value between 180 and 86,400 seconds can be set.

-i adpname -l linkspeed

Sets a link speed. One of the following values can be set:

1G|1g : Sets 1 Gbps.


75


AUTO|auto : Sets a link speed with AUTO.

-i adpname -t topology

Sets a topology. One of the following values can be set:

NPORT|nport : Makes an NPORT connection.

Connects to the Fibre Channel switch.

AL|al : Sets the FC-AL topology.

AUTO|auto : Sets a topology automatically.

-i adpname -v

Displays the settings as listed below.

Item Value Description

boot function DISABLE/ENABLE Enables or disables the boot function.

Target_ID Example) 0 (hexadecimal) Bound Target_ID

Target WWN Example) 210000e0004101d9(hexadecimal)

Bound target

Target DID Example) 010111 (hexadecimal) Bound DID

topology AL/N_Port/AUTO Set topology. AUTO refers to an automaticallyset topology.

link-speed 1G/2G/4G/AUTO Set link speed. AUTO refers to an automaticallyset link speed.

boot wait time DISABLE or numeric value(decimal)

Set boot wait time. Value is specified in seconds.DISABLE overrides a boot wait time.

interval time DISABLE This item is not available and cannot be changed.

boot wait msg DISABLE This item is not available and cannot be changed.

76

-i adpname -s savefile

Saves the settings as listed below.





Bound target


topology AL/N_Port/AUTO Set topology. AUTO refers to an automaticallyset topology.

link-speed 1G/2G/4G/AUTO Set link speed. AUTO refers to an automaticallyset link speed.


Set boot wait time. Value is specified in seconds.DISABLE overrides a boot wait time.

interval time DISABLE This item is not available and cannot be changed.

boot wait msg DISABLE This item is not available and cannot be changed.

-i adpname -r|-R filename

Updates the boot code on the Fibre Channel card with the settings saved with -s.

If -r is specified, the boot code on the Fibre Channel card is updated with all settings, except forthe boot function.

If -R is specified, the boot code on the Fibre Chanel card is updated with all settings, including theboot function.

-i adpname -c conffile

Updates the boot code on the Fibre Channel card with the settings of the driver setting file(/kernel/drv/fjpfca.conf).

-i adpname -C [-y]

Erases all settings, except for enabling/disabling of the boot function.

If -y is not attached, a message asking if you are sure you want to erase the settings is displayed.

If -y is attached, the settings are erased unconditionally.

77

-i adpname -b ENABLE|DISABLE

Enables or disables the Fibre Channel card boot function. One of the following values can be set:

ENABLE : Enables the boot function.

DISABLE : Disables the boot function.

Note

Enabling or disabling the boot function on either port of the dual-channel 4Gbps Fibre Channelcard (SE0X7F12x) will automatically impart the same setting to the other port. It is not possibleto make a change to the setting of either port alone and not both.

Example

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -f 0 -P 0x210000e0001014d9 <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -f 1 -I 0x10c00 <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -d 0 <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -D <RETURN>

delete all bind registration ? [y(Y),n(N) ] y <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -w 180 <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -l 4g <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -t nport <RETURN>


boot function : ENABLE

topology : N_Port

link-speed : 4G

boot wait time : 180 ( interval time : DISABLE , boot wait msg : DISABLE )

bind-target: Target_ID=0,WWPN=0x210000e0001014d9

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -s savefile <RETURN>

# /usr/sbin/FJSVpfca/fc_hbaprp -i fjpfca0 -r savefile <RETURN>



78

A.2 Command Executable on the OBPBefore starting the procedure, set the to maintenance mode and restart the server. If you use SPARCEnterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute the following command:



If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switch toservice mode and execute the following command:


If an attempt is made to configure the Fibre Channel card in any other condition, it could hang up. Whenit does, turn the power to the server off, and then turn it back on.

To carry out this command, you need to move to the node of the Fibre Channel card on which toconfigure.

Example: single-channel 4Gbps Fibre Channel card (SE0X7F11x) and dual-channel 4Gbps FibreChannel card (SE0X7F12x) mounted on a server

ok show-devs <RETURN>

/pci@1,700000

/pci@2,600000

**

/openprom

/chosen

/packages

/pci@1,700000/fibre-channel@0 *physical path name of the single channel 4Gbps FibreChannel card

/pci@2,600000/fibre-channel@0 *physical path name of the dual-channel 4Gbps FibreChannel card port0

/pci@2,600000/fibre-channel@0,1 physical path name of the dual-channel 4Gbps FibreChannel card port1


ok

After configuring the Fibre Channel, release the service mode of the server, if necessary.

If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute the followingcommand;


79

If you use, SPARC Enterprise M3000/M4000/M8000/M9000, set the server mode switch back to"AUTO".

1 fjpfca-set-bootfunction

Name

fjpfca-set-bootfunction

Format

ENABLE | DISABLE fjpfca-set-bootfunction

Function

Enables or disables the Fibre Channel card boot function.

Note) The server must be restarted or the reset-all command must be carried out after thiscommand has been carried out. In configuring on more than one card as well, remember to carryout reset-all for every card mounted.

Operands

One of the following values can be set:

ENABLE : Enable the boot function.

DISABLE : Disables the boot function.

Note

Enabling or disabling the boot function on either port of the dual-channel 4Gbps Fibre Channelcard (SE0X7F12x) will automatically impart the same setting to the other port. It is not possibleto make a change to the setting of either port alone and not both.

Example

ok ENABLE fjpfca-set-bootfunction <RETURN>


..

ok DISABLE fjpfca-set-bootfunction <RETURN>


2 fjpfca-output-prop

Name

fjpfca-output-prop

80

Format

fjpfca-output-prop

Function

Displays the settings stored in ROM on the Fibre Channel card.





Bound target


topology AL/N_Port/AUTO Set topology. AUTO refers to anautomatically set topology.

link-speed 1G/2G/4G/AUTO Set link speed. AUTO refers to anautomatically set link speed.


Set boot wait time. Value is specifiedin seconds.DISABLE overrides a boot wait time.

interval time DISABLE This item is not available and cannotbe changed.

boot wait msg DISABLE This item is not available and cannotbe changed.

Example


boot function : ENABLE

topology : AUTO

link-speed : AUTO


bind-target: Target_ID=0,WWPN=0x210000e0001014d9

81

3 fjpfca-set-linkspeed

Name

fjpfca-set-linkspeed

Format

1g | 2g | 4g | auto fjpfca-set-linkspeed

Function

Sets a link speed.

Operands


1g : Sets 1 Gbps.

2g : Sets 2 Gbps.

4g : Sets 4 Gbps.

auto: Sets a link speed automatically.

Example




ok auto fjpfca-set-linkspeed <RETURN>

Default value

auto

Note

This command is operable only if the boot function is enabled.

4 fjpfca-set-topology

Name

fjpfca-set-topology

Format

nport | al | auto fjpfca-set-topology

Function

82

Sets a topology.

Operands

nport: Makes an NPORT connection.Connects to the Fibre Channel switch.

al: Sets the FC-AL topology.

auto: Sets a topology automatically.

Example

ok nport fjpfca-set-topology <RETURN>

ok al fjpfca-set-topology <RETURN>

ok auto fjpfca-set-topology <RETURN>

Default value

auto

Note


5 fjpfca-bind-target

Name

fjpfca-bind-target

Format

value1 target-alpa | target-did |target-wwpn value2 fjpfca-bind-target

Function

Configure a target device. Up to 10 entries can be registered.

Operands

Specify a target device to be connected with either Port_ID (DID) or WWPN.

value1: Specifies Target_ID of the target device with a hexadecimal number.

target-wwpn: Specifies WWPN of the target device (boot device specification by WWPN).

target-alpa: Specifies Port_ID (DID) of the target device (boot device specification byPort_ID).

It uses when FC_AL is connected.

target-did: Specifies Port_ID (DID) of the target device (boot device specification by

83

Port_ID).

It uses when FC_SWITCH is connected.

value2: Specifies Port_ID (DID or WWPN with a hexadecimal number.

Example

ok 0 target-wwpn 210000e0004101d9 fjpfca-bind-target <RETURN> * WWN specification

ok 1 target-alpa 11206 fjpfca-bind-target <RETURN> * Port_ID(DID) specification

ok 2 target-did 11000 fjpfca-bind-target <RETURN> * Port_ID(DID) specification

Note


6 led-flash

Name

led-flash

Format

[sec-time] led-flash

Function

Flashes the LED on the Fibre Channel card (for 10 seconds by default and for 60 seconds at thelongest).

Use this command to verify the location of the Fibre Channel card and where the ports arepositioned (on a dual-channel 4Gbps Fibre Channel card).

Operands

Specify the duration of time in which the LED flashes.

A number preceded by d# is assumed to be a decimal number. All other numbers are assumed tobe hexadecimal.

Example

ok led-flash Flashes the LED for 10 seconds.

ok d# 10 led-flash Flashes the LED for 10 seconds (period specified with a decimal number).

ok 3c led-flash Flashes the LED for 60 seconds (period specified with a hexadecimal number)

Note


84

7 fjpfca-set-boot-wait-time

Name

fjpfca-set-boot-wait-time

Format

wait-time | DISABLE fjpfca-set-boot-wait-time

Function

If a power supply interlock control is implemented between the server and a disk array device, it isnecessary to let the disk array device start up before launching a boot sequence. Use the fjpfca-set-boot-wait-time command to delay the launch of the boot sequence for a specified period oftime.

The Fibre Channel card is monitoring the status of the disk array device even while the OS waitsto be booted, ready to start booting automatically as soon as it confirms that the disk array deviceis seen started up, even before the specified period of time expires.

The period of time for which the boot sequence is delayed can be set in seconds between 180 and86,400 seconds.

The command comes with this mode disabled by default.

Referring to the manual supplied with the disk array device, set (for the boot wait time) the amountof time that it takes for the system to enter the READY state after the POWER switch is pressed.

Operands

Specify a wait time (in seconds) at boot time with a hexadecimal number.

A number preceded by d# is assumed a decimal number.

DISABLE overrides a boot wait time.

Example

ok d# 1200 fjpfca-set-boot-wait-time Sets 1200 seconds

ok b4 fjpfca-set-boot-wait-time Sets 180 seconds

ok DISABLE fjpfca-set-boot-wait-time Overrides a boot wait time

Default value

DISABLE

Note


85

8 fjpfca-info

Name

fjpfca-info

Format

STATUS | PROBE fjpfca-info

Function

Displays connectivity information about the Fibre Channel card mounted.

Operands


STATUS Displays the Link status of the Fibre Channel card.Indicates whether the target device set up on the Fibre Channel card is connectable.

PROBE Displays a list of target devices connectable from the Fibre Channel card.

Example

ok STATUS fjpfca-info <RETURN>

Link_status=up topology=Nport port_id=0x010000 wwpn=1000000b5d65c00a(0)

port_id=0x010100 tid=0 wwpn=210000e00004101d9 connected(0)


Target -- DID 10100 WWPN 210000e0004101d9 FUJITSU-E4000-0000

Note


9 fjpfca-target-cancel

Name

fjpfca-target-cancel

Format

tgt_id fjpfca-target-cancel

Function

Erases the settings of a target device.

86

Operands

tgt_id Specifies Target_ID of the target device with a hexadecimal number.

Example

ok 0 fjpfca-target-cancel <RETURN>

Note


10 fjpfca-all-target-cancel

Name

fjpfca-all-target-cancel

Format

fjpfca-all-target-cancel

Function

Erases all settings of the target device.

Example

ok fjpfca-all-target-cancel <RETURN>

delete all bind registration ? [ y(Y),n(N) ] y

Note


87

Appendix B Checking the Fibre ChannelCard Boot Code VersionNumber

This appendix explains how to confirm the boot code (firmware) version number of the Fibre Channelcard.

There are two ways to perform this confirmation: confirming the code by checking the OS and confirmingthe code by checking OBP.

B.1 Checking on the OSPerform this procedure with the OS already booted and with FUJITSU PCI Fibre Channel 4.0 or anynewer package installed.

View /var/adm/messages to check the boot code version number from the following display:

scsi: [ID 243001 kern.info] /pci@1,700000/fibre-channel@0 (fjpfca0):

INFO : FUJITSU PCI Fibre Channel FCode Version : v12l30, boot_function=ENABLE;

B.2 Checking on the OBPPerform this procedure on the OBP. Before starting the procedure, set the to maintenance mode andrestart the server. If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440,execute the following command:



If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switch toservice mode and execute the following command:


Move to the node of the Fibre Channel card whose boot code version number is to be confirmed, andexecute the .properties command.

88

Read the value of fjpfca_fcode_vl.


ok .properties <RETURN>

status okay

fru PCI slot(PCI#08)

component-name PCI#08

assigned-addresses 81001814 00000000 00000700 00000000 00000100

(Omission)

fjpfca_fcode_vl v12l30

(Omission)

ok

After confirming the WWN (World Wide port Name) of the Fibre Channel Card, release the servicemode of the server, if necessary.

If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute the followingcommand;


If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switch back to"AUTO".

89

Appendix C Recording SAN Boot SettingInformation

Record and save to any other document the values listed in the "SAN Boot Setting Information" table thathave been set at installation. Whenever a Fibre Channel card in use is replaced, reconfigure the same setof values of this "SAN Boot Setting Information" on the replacement card as well.

The reconfiguration sequence takes different courses depending on which of the following cases applies:

1. Active replacement of the card during OS running on an alternate path.2. Cold replacement of the card when OS is available on alternate path.3. Cold replacement of the card when OS can not boot on any paths.

When a card is replaced in Cases 1 and 2 above, the SAN boot setting information can be reconfigureinto the new card from the Fibre Channel driver environment definition files using the fc_hbaprpcommand. This "SAN Boot Setting Information" is required when replacing a card in Case 3. In thiscase, carry out a command executable on the OBP to reconfigure the SAN boot setting information.

Keep a record of this setting information with regard to all Fibre Channel cards used for rebooting the OS.

90

"SAN Boot Setting Information"

No. Item Explanation Actual value (record)

1 Device path name Name of a physical device path on theOS (/pci@XXXX/yyyy@z)

2 Slot position Mounted slot position

3 Boot function (boot function) Specify whether to enable or disable theboot function.

4 Topology information(topology)

Specify a topology (nport to connect tothe switch, al to make a directconnection)

5 Link speed (link speed) Use to fix a speed of transmission line(1G/2G/4G/auto).

6 Boot delay function (boot waittime)

Specify whether to enable or disable theboot delay function and a boot waittime.

7 Target bind information(Target_ID/TargetWWN|Target DID)

Specify disk array device bindinformation (for all registered entries). 1

target_id :

wwn | did :

2target_id :

wwn | did :

3target_id :

wwn | did :

4target_id :

wwn | did :

5target_id :

wwn | did :

6target_id :

wwn | did :

7target_id :

wwn | did :

8target_id :

wwn | did :

9target_id :

wwn | did :

10target_id :

wwn | did :

91

Appendix D Making Fixes to Setting Filesafter a Boot Failure

Errors in any SAN Boot setting file (such as sd.conf, mplb.conf or /etc/system) could prevent the OSfrom starting up after a boot failure. When this occurs, start the OS in single-user mode and mount thesystem disk on a disk array device to make fixes to the setting file as instructed in this appendix.

The way the OS is started depends on which of the following ways has been used to install the OS on thesystem disk on a disk array device.

System disk installation as per Section 4.1.1, "Creating a boot disk using a network install server"

System disk installation as per Section 4.1.2, "Creating a boot disk by copying an existing boot diskresiding on an internal disk"

This chapter assumes that Solaris is installed on disk device c7t16d0 on a disk array device.

D.1 If the OS Has Been Installed As Per Section4.1.1, "Creating a boot disk using a networkinstall server"






2. Boot from the network. Start the OS in single-user mode.


3. Mount the system disk on a disk array device.For the disk to be mounted, specify the device that has been set as an OS installation device inSection "4.1.1.4, Configuring Custom JumpStart"


4. Fix the setting files in the /mnt directory.Example) If /etc/vfstab is the cause of the failure and requires change, fix /mnt/etc/vfstab.

92

5. If you use Solaris 10 10/08 or newer, update the boot-archive. If you use Solaris 10 5/08 or older,proceed to the next step.

# /mnt/sbin/bootadm update-archive -R /mnt <RETURN>

6. When the fix is completed, unmount the system disk and migrate to the OBP environment asinstructed below.

# cd / <RETURN>

# umount /mnt <RETURN>

7. Release the service mode of the server, if necessary.If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute thefollowing command;


If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switchback to "AUTO".

8. Perform a retry from the boot sequence afterwards.

D.2 If the OS Has Been Installed As Per Section4.1.2, "Creating a boot disk by copying anexisting boot disk residing on an internaldisk"






2. Boot from the internal disk. Start the OS in single-user mode.

ok boot disk0 -s <RETURN>

^^^^^^

^^^^^^ denotes an internal disk specification.

3. Mount the system disk on a disk array device.

93

For the disk to be mounted, specify the disk that has been verified in Section "4.1.2.1 Gettingready to copy the boot disk to a disk array device."


4. Fix the setting files in /mnt directory.Example) If /etc/vfstab is the cause of the failure and requires change, fix /mnt/etc/vfstab.

5. If you use Solaris 10 10/08 or newer, update the boot-archive. If you use Solaris 10 5/08 or older,proceed to the next step.

# /mnt/sbin/bootadm update-archive -R /mnt <RETURN>

6. When the fix is completed, unmount the system disk and migrate to the OBP environment asinstructed below.

# cd / <RETURN>

# umount /mnt <RETURN>

7. Release the service mode of the server, if necessary.If you use SPARC Enterprise T1000/T2000/T5120/T5140/T5220/T5240/T5440, execute thefollowing command;


If you use SPARC Enterprise M3000/M4000/M5000/M8000/M9000, set the server mode switchback to "AUTO".

8. Perform a retry from the boot sequence afterwards.

94

Appendix E Fibre Channel Driver/BootCode Auto-Target BindingFunctions

This appendix describes the Fibre Channel driver/boot code auto-target binding functions.

E.1 Fibre Channel Driver Auto-Target BindingFunction

The Fibre Channel driver auto-target binding function enables the Fibre Channel driver to automaticallyconnect a server to target devices without requiring their definitions in fcp-bind-target in fjpfca.conf.

The Fibre Channel driver connects all target devices attached to a fabric device to the lowest availableTarget_ID in ascending order of WWNs.

1. Fibre Channel driver auto-target bind example

Server

FC Card

FC Switch

ETERNUS #2

CM0 CM1

ETERNUS #1

CM0 CM1

ETERNUS #1 CM0(WWN) : 0x210000e0004101d9CM1(WWN) : 0x230000e0004101d9

ETERNUS #2 CM0(WWN) : 0x210000e0004101daCM1(WWN) : 0x230000e0004101da

Target_ID:0Target_ID:3

Target_ID:2 Target_ID:1

95

In the example shown above, the Fibre Channel driver automatically connects target disk array devices inascending order of WWNs as follows:

Connect ETERNUS #1 CM0(WWN=0x210000e0004101d9) to Target_ID:0

Connect ETERNUS #1 CM1(WWN=0x230000e0004101d9) to Target_ID:2

Connect ETERNUS #2 CM0(WWN=0x210000e0004101da) to Target_ID:1

Connect ETERNUS #2 CM1(WWN=0x230000e0004101da) to Target_ID:3

Prerequisites for implementing this automatic connectivity are:

1. "fcp-auto-binding function=1;" is entered in fjpfca.conf.2. SAN Boot environment built on a fabric connection

The Fibre Channel driver auto-target binding function is designed with primary emphasis on building aSAN Boot environment effortlessly, and its use is recommended only for the purpose of building anenvironment.

If the Fibre Channel driver auto-target binding function is used in normal operations, it might fail tocomplete intended target device connections owing to the effect of the failure of target devices or the like.The use of fcp-bind-target to make target device connections is recommended for normal operations.

For information about marking target device connections using fcp-bind-target, refer to the "FUJITSUPCI Fibre Channel Guide."

96

E.2 Fibre Channel Boot Code Auto-TargetBinding Function

This section describes the Fibre Channel boot code auto-target binding function.

The Fibre Channel boot code auto-target binding function lets the Fibre Channel boot code detect targetdevices automatically without requiring their definitions in fjpfca-bind-target, allowing only those targetdevices having the lowest Port-ID assigned to them to be connected to a server for SAN Boot.

1. Fibre Channel boot code auto-target bind example

Server

FC Card

ETERNUS #1

CM0 CM1

Port_ID:10100 Port_ID:10400

Port_ID:10200

Port_ID:10300

ETERNUS #2

CM0 CM1

FC Switch

Port_ID:10000

Connected as targetdevices to implementSAN Boot

In the example shown above, target devices located at ETERNUS#1 CM0 with the lowest assignedPort_ID are connected to implement SAN Boot.

Prerequisites to implementing this automatic connectivity are:

1. No target definitions are given in fjpfca-bind-target2. SAN Boot environment built on a fabric connection

Because the Fibre Channel boot code auto-target binding function focuses on effortless implementation ofSAN Boot, it use is recommended only in environment construction and in environments in which FCswitch-based zoning is implemented. If the Fibre Channel boot code auto-target binding function is usedelsewhere, it might fail to complete intended target device connections owing to the effect of the failureof target devices or the like. The use of fjpfca-bind-target to manually configure target deviceconnections is recommended in incompatible environments. For information about making target deviceconnections using fjpfca-bind-target, see Appendix A, "Boot Device Setup Commands."

If SAN Boot is implemented with the boot code auto-target binding function, the target deviceinformation is imparted to the Fibre Channel driver, enabling it to make target device connectionsautomatically (fcode-auto-bind function). For more information about the fcode-auto-bind function, referto "FUJITSU PCI Fibre Channel Guide."

97

Appendix F SAN Boot release procedure

If you release the multipath definition of the boot disk, it does the following procedures.

It becomes impossible might do boot when it releases by the methods other than the procedure or itmakes a mistake in the procedure.

After the mirroring is released, the following procedures are executed when mirroring it withPRIMECLUSTER GDS.

F.1 ETERNUS Multipath DriverIt is ETERNUS multipath driver's release procedure.

All multipath are released.

1. Edit /etc/vfstab file to rewrite mount point.

/dev/FJSVmplb/dsk/mplb0s0 /dev/FJSVmplb/rdsk/mplb0s0

↓ ↓

/dev/dsk/c2t16d0s0 /dev/rdsk/c2t16d0s0

Former mount point is either path constituting multipath displayed by the iompadm command.

# /usr/opt/FJSViomp/bin/iompadm info /dev/FJSVmplb/fiomp/adm0 <RETURN>

IOMP: /dev/FJSVmplb/fiomp/adm0

Element:

/dev/rdsk/c2t16d0s2 online active block "good status with active

[E30004641- 130011-CM01-CA01-PORT36] (mplbt0)"

/dev/rdsk/c3t16d0s2 online standby block "good status with standby

[E30004641- 130011-CM00-CA00-PORT32] (mplbt32)"

2. Release the multipath.

# mplbconfig -r <RETURN>

Cannot unload module: mplb

Will be unloaded upon reboot.

Forcing update of mplb.conf.

3. Edit /kernel/drv/mplbt.conf file to delete all the definitions.Example）The following delete.

98

name="mplbt" parent="fjpfca" target=16 lun=0;

…

4. Edit /kernel/drv/mplbh.conf file and make it default setting.Example）Delete all the following definitions of the mplbh.conf file.

mplbh-path-0="pci10cf,1178-0-10" mplbh-path-1="pci10cf,1178-1-10"

mplbh-disk-name="E30004641- 130011-0010";

mplbh-detect-disk-num=1;

mplbh-detect-disk-0="E30004641- 130011-0010";

mplbh-used-path-num=2;

mplbh-used-path-0="pci10cf,1178-0-10";

mplbh-used-path-1="pci10cf,1178-1-10";

It adds when there is no ";" at the end of the following lines.

name="mplbh" parent="mplbx" instance=X;

5. Edit the /kernel/drv/sd.conf file to delete the definition of mplb.Delete the following lines.

# Start eternusmpd configuration -- do NOT alter or delete this line

name="sd" parent="mplbh" target=0 lun=0;

…

# End eternusmpd configuration -- do NOT alter or delete this line

6. Delete the following definitions added to /etc/system file.Delete the following lines.



forceload: drv/mplb

forceload: drv/sd

7. Stop the service.

# svcadm disable -t svc:/system/fjsvmplb:default <RETURN>

8. Make a special file of sd. Ignore the error message.

99


Cannot unload module: sd

Will be unloaded upon reboot.

Forcing update of sd.conf.

9. Restart the server.


# reboot <RETURN>

10. When the dump device changed, it returns.

# dumpadm -d /dev/dsk/c2t16d0s3 <RETURN>

100

Appendix G Updating the Fibre ChannelCard Boot Code

This appendix explains how to update the boot code (firmware) of the Fibre Channel card.

It explains the procedure for updating the boot code to V12L50 as an example. To confirm the boot codeversion please see Appendix B, "Checking the Fibre Channel Card Boot Code Version Number."

1. Apply the driver patch

The FJPFC driver patch necessary for the update of the boot code is applied to the system volume formaintenance. Apply the 914583-08 or newer patch.

The system volume for maintenance indicates the system volume on the server for the network boot orthe system volume on the internal disk.

When the system volume for maintenance is on the internal disk

(1-1) Applies patch to the internal disk.

# patchadd 914583-08 <RETURN>

(1-2) Migrate the system to the OBP environment.


(1-3) Start the OS from the system volume for maintenance.

ok boot disk0 -s <RETURN>

^^^^^^

^^^^^^ denotes an internal disk specification.

When the system volume for maintenance is on the network install server

- Solaris 10 5/08 or older

(1-1) Apply the patch in the OS install image on the install server.

(INSTALL SERVER) # patchadd -C/export/install/Solaris10_hostname/Solaris_10/Tools/Boot/ 914583-08 <RETURN>



101

- Solaris 10 10/08 or newer

(1-1) Create the working directory for unpacking miniroot.

(INSTALL SERVER) # mkdir /tmp/work <RETURN>

(1-2) Unpack the miniroot to the work directory using the root_archive(1M) command.

If /tmp/work/tmp/AdDrEm.lck file is not exist, following procedure ignoring.


(INSTALL SERVER) # rm /tmp/work/tmp/AdDrEm.lck <RETURN>

Note) The following messages might be displayed when root_archive command is executed. Youmight ignore these messages.





(1-3) Apply the patch in the working directory.

(INSTALL SERVER) # patchadd -C /tmp/work 914583-08 <RETURN>

(1-4) Pack the working directory.


(INSTALL SERVER) # /tmp/work/boot/solaris/bin/root_archive packmedia /tmp/media/tmp/work <RETURN>


(1-5) Copy the file in /tmp/media directory to the installation image on the install server.

Target device path name for "umount -f" and "lofiadm -d" commands can confirm by "df -k"command.





102




2. Update the boot code.

After OS in the system volume for maintenance starts, execute the update command.

# /usr/sbin/FJSVpfca/fc_fcode_update <RETURN>

This command updates the boot code to all cards mounted in the server. In the case of the latestboot cord, it is not updated.

Example

# /usr/sbin/FJSVpfca/fc_fcode_update <RETURN>

Are you ready to udpate FCode? [y,n] yes <RETURN>

Overwrite FCode to fjpfca0 (V12L40 -> V12L50).. Success

fjpfca1: Same version of FCode has already installed. (current version :V12L50)

Note)

- The setting of the card is kept by update.

- Do not execute it while execute I/O.

Umount is done beforehand when using it as a filesystem.

- There is no problem even if the optical cable is connected.

- If the following messages are displayed when the command is executed, the Fibre Channelcard might be defective. In this case, please exchange the cards for a new one.

WARNING: /pci@%x,%x/fibre-channel@%x (fjpfca%d):

ROMAccess failed.



Manual Update HistoryDate published Edition Description

May, 23rd, 2007 1.0 First release

Sep, 14th, 2007 1.1 Support ETERNUS 2000 series

Support SPARC Enterprise T5120/T5220

Oct, 14th, 2008 1.2 Support Solaris 10 10/08

Support SPARC Enterprise M3000/T5140/T5240/T5440

Apr, 3rd, 2009 1.3 Change the restore procedure

Change the check command procedure of Custom JumpStart

Oct, 28th, 2009 1.4 Add the Fibre Channel card boot code update procedure

Add the precaution 18

Support ETERNUS DX series

May, 15th, 2012 1.5 Support ETERNUS DX90

Add the precaution 16, 17, 18, and 19

Mar, 13th, 2013 1.6 Add the precaution 19

Apr, 11th, 2014 1.7 Change the Preface and Chapter 3 Precautions

SPARC Enterprise - Fujitsu · Fujitsu attaches to the damage caused by not the one that the...

Documents

Transcript of SPARC Enterprise - Fujitsu · Fujitsu attaches to the damage caused by not the one that the...